Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a test case in CitationKeyGeneratorTest.java file #12194

Closed
wants to merge 43 commits into from
Closed
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
c407a30
Added a test case in CitationKeyGeneratorTest.java file
yoyounik Nov 16, 2024
581fb61
removed the wrong import
yoyounik Nov 16, 2024
8f66b40
Updated the pattern
yoyounik Nov 17, 2024
df2d656
Reorganized import
yoyounik Nov 17, 2024
f2559e5
Remove LaTeX commands (e.g., \mkbibquote{}) using regex.
yoyounik Nov 17, 2024
f845041
Updated with RemoveLatexCommandsFormatter in CitationKeyGenerator.java
yoyounik Nov 18, 2024
dd1f97f
Added the import for RemoveLatexCommandsFormatter
yoyounik Nov 18, 2024
3b1eda2
Added imports for removing OpenRewrite issue
yoyounik Nov 18, 2024
784f756
updated code
yoyounik Nov 18, 2024
19a80c3
updated with the import
yoyounik Nov 18, 2024
a5a37b9
updated with the import
yoyounik Nov 18, 2024
6c4e0a8
updated with the import
yoyounik Nov 18, 2024
beacb03
Removed the comment
yoyounik Nov 19, 2024
0368236
Updated with imports
yoyounik Nov 19, 2024
b70ad43
Updated with imports
yoyounik Nov 19, 2024
8e4a7e1
Updated with imports
yoyounik Nov 19, 2024
6fff450
Updated with imports
yoyounik Nov 19, 2024
85befed
Merge remote-tracking branch 'origin/fix/strip-latex-from-filename' i…
yoyounik Nov 20, 2024
07064db
Instantiating the formatter as object variable
yoyounik Nov 20, 2024
c53891f
Instantiating the formatter as object variable
yoyounik Nov 20, 2024
4868e0d
Merge remote-tracking branch 'origin/fix/strip-latex-from-filename' i…
yoyounik Nov 20, 2024
912e1b2
Merge remote-tracking branch 'origin/fix/strip-latex-from-filename' i…
yoyounik Nov 20, 2024
74dc490
Merge remote-tracking branch 'origin/fix/strip-latex-from-filename' i…
yoyounik Nov 20, 2024
f1c8535
Merge remote-tracking branch 'origin/fix/strip-latex-from-filename' i…
yoyounik Nov 20, 2024
97f968b
Merge remote-tracking branch 'origin/fix/strip-latex-from-filename' i…
yoyounik Nov 20, 2024
4899a18
Merge remote-tracking branch 'origin/fix/strip-latex-from-filename' i…
yoyounik Nov 20, 2024
28f9f2f
Merge remote-tracking branch 'origin/fix/strip-latex-from-filename' i…
yoyounik Nov 20, 2024
ff7925f
Updated with changes in LatexCleanupFormatter.java
yoyounik Nov 21, 2024
d41a9d7
Updated with changes in LatexCleanupFormatter.java
yoyounik Nov 21, 2024
3c00c29
Updated cleanKey method.
yoyounik Nov 22, 2024
8d9845b
Updated cleanKey method.
yoyounik Nov 22, 2024
83c5841
Updated cleanKey method.
yoyounik Nov 22, 2024
89b1106
Updated cleanKey method.
yoyounik Nov 22, 2024
d95e7e1
Merge remote-tracking branch 'origin/fix/strip-latex-from-filename' i…
yoyounik Nov 22, 2024
83e3c06
Updated cleanKey method.
yoyounik Nov 22, 2024
6ecdb03
Updated cleanKey method.
yoyounik Nov 22, 2024
9cc84a1
Updated cleanKey method.
yoyounik Nov 22, 2024
36a42f3
Updated cleanKey method.
yoyounik Nov 22, 2024
25ce9a9
Updated cleanKey method.
yoyounik Nov 22, 2024
0d2bea3
Updated cleanKey method.
yoyounik Nov 27, 2024
ab938b0
Updated
yoyounik Nov 27, 2024
89d2f05
Updated
yoyounik Nov 27, 2024
3e69e0c
Updated
yoyounik Nov 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@
import java.util.function.Function;
import java.util.regex.PatternSyntaxException;

import org.jabref.logic.layout.format.RemoveLatexCommandsFormatter;
import org.jabref.logic.formatter.bibtexfields.LatexCleanupFormatter;
import org.jabref.model.entry.field.StandardField;
import org.jabref.model.FieldChange;
import org.jabref.model.database.BibDatabase;
import org.jabref.model.database.BibDatabaseContext;
Expand Down Expand Up @@ -44,6 +47,7 @@ public class CitationKeyGenerator extends BracketedPattern {
private final BibDatabase database;
private final CitationKeyPatternPreferences citationKeyPatternPreferences;
private final String unwantedCharacters;
private final RemoveLatexCommandsFormatter formatter = new RemoveLatexCommandsFormatter();

public CitationKeyGenerator(BibDatabaseContext bibDatabaseContext, CitationKeyPatternPreferences citationKeyPatternPreferences) {
this(bibDatabaseContext.getMetaData().getCiteKeyPatterns(citationKeyPatternPreferences.getKeyPatterns()),
Expand Down Expand Up @@ -91,7 +95,11 @@ public static String removeUnwantedCharacters(String key, String unwantedCharact
}

public static String cleanKey(String key, String unwantedCharacters) {
return removeUnwantedCharacters(key, unwantedCharacters).replaceAll("\\s", "");
String cleanedKey = removeUnwantedCharacters(key, unwantedCharacters).replaceAll("\\s", "");
if (cleanedKey.startsWith("-")) {
cleanedKey = cleanedKey.substring(1);
}
return cleanedKey;
}

/**
Expand All @@ -102,10 +110,15 @@ public static String cleanKey(String key, String unwantedCharacters) {
*/
public String generateKey(BibEntry entry) {
Objects.requireNonNull(entry);

LatexCleanupFormatter formatter = new LatexCleanupFormatter();
String formattedTitle = formatter.format(entry.getField(StandardField.TITLE).orElse(""));
entry.setField(StandardField.TITLE, formattedTitle);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong! The entry MUST NOT be modified.

I think, you can just delete these two lines.


String currentKey = entry.getCitationKey().orElse(null);

String newKey = createCitationKeyFromPattern(entry);
newKey = replaceWithRegex(newKey);
newKey = formatter.format(newKey);
newKey = appendLettersToKey(newKey, currentKey);
return cleanKey(newKey, unwantedCharacters);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,10 @@
* Simplifies LaTeX syntax. {@see org.jabref.logic.layout.format.RemoveLatexCommandsFormatter} for a formatter removing LaTeX commands completely.
*/
public class LatexCleanupFormatter extends Formatter {

private static final Pattern REMOVE_REDUNDANT = Pattern
.compile("(?<!\\\\[\\p{Alpha}]{0,100}\\{[^\\}]{0,100})\\}([-/ ]?)\\{");
.compile("(?<!\\\\[\\p{Alpha}]+\\{)\\}([-/ ]?)\\{");

private static final Pattern REPLACE_WITH_AT = Pattern.compile("(^|[^\\\\$])\\$");
private static final Pattern REPLACE_WITH_AT = Pattern.compile("(^|[^\\\\])\\$(?!\\$)");
private static final Pattern REPLACE_EVERY_OTHER_AT = Pattern.compile("([^@]*)@@([^@]*)@@");
private static final Pattern MOVE_NUMBERS_WITH_OPERATORS = Pattern.compile("([0-9\\(\\.]+[ ]?[-+/]?[ ]?)\\$");
private static final Pattern MOVE_NUMBERS_RIGHT_INTO_EQUATION = Pattern.compile("@@([ ]?[-+/]?[ ]?[0-9\\)\\.]+)");
Expand Down Expand Up @@ -53,7 +52,9 @@ public String format(String oldString) {

newValue = ESCAPE_PERCENT_SIGN_ONCE.matcher(newValue).replaceAll("$1\\\\%"); // escape %, but do not escapee \% again, used for comments in TeX

return newValue;
newValue = newValue.replace("mkbibquote", "");

return newValue.trim();
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,68 +4,53 @@
import org.jabref.model.strings.StringUtil;

public class RemoveLatexCommandsFormatter implements LayoutFormatter {

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was there a need to touch this class?

@Override
public String format(String field) {
StringBuilder cleanedField = new StringBuilder();
StringBuilder currentCommand = null;
char currentCharacter;
boolean escaped = false;
boolean incommand = false;
int currentFieldPosition;
for (currentFieldPosition = 0; currentFieldPosition < field.length(); currentFieldPosition++) {
currentCharacter = field.charAt(currentFieldPosition);
if (escaped && (currentCharacter == '\\')) {
cleanedField.append('\\');
escaped = false;
// \\ --> first \ begins the command, second \ ends the command
// \latexommand\\ -> \latexcommand is the command, terminated by \, which begins a new command
incommand = false;
} else if (currentCharacter == '\\') {

for (int currentFieldPosition = 0; currentFieldPosition < field.length(); currentFieldPosition++) {
char currentCharacter = field.charAt(currentFieldPosition);

if (currentCharacter == '\\') {
// Start of a LaTeX command
escaped = true;
incommand = true;
currentCommand = new StringBuilder();
} else if (!incommand && ((currentCharacter == '{') || (currentCharacter == '}'))) {
// Swallow the brace.
} else if (Character.isLetter(currentCharacter) || StringUtil.SPECIAL_COMMAND_CHARS.contains(String.valueOf(currentCharacter))) {
escaped = false;
if (incommand) {
currentCommand.append(currentCharacter);
if ((currentCommand.length() == 1)
&& StringUtil.SPECIAL_COMMAND_CHARS.contains(currentCommand.toString())) {
// This indicates that we are in a command of the type \^o or \~{n}
incommand = false;
escaped = false;
}
} else {
cleanedField.append(currentCharacter);
}
} else if (Character.isLetter(currentCharacter)) {
escaped = false;
if (incommand) {
// We are in a command, and should not keep the letter.
} else if (incommand) {
// Collect characters for the LaTeX command name
if (Character.isLetter(currentCharacter) ||
StringUtil.SPECIAL_COMMAND_CHARS.contains(String.valueOf(currentCharacter))) {
currentCommand.append(currentCharacter);
} else {
cleanedField.append(currentCharacter);
}
} else {
if (!incommand || (!Character.isWhitespace(currentCharacter) && (currentCharacter != '{'))) {
cleanedField.append(currentCharacter);
} else {
if (!Character.isWhitespace(currentCharacter) && (currentCharacter != '{')) {
// do not append the opening brace of a command parameter
// do not append the whitespace character
cleanedField.append(currentCharacter);
}
if (incommand) {
// eat up all whitespace characters
while (currentFieldPosition + 1 < field.length() && Character.isWhitespace(field.charAt(currentFieldPosition + 1))) {
currentFieldPosition++;
} else if (currentCharacter == '{') {
// Found the start of the command's parameters; skip the command name
int braceLevel = 1;
currentFieldPosition++; // Move past the opening brace
while (currentFieldPosition < field.length() && braceLevel > 0) {
currentCharacter = field.charAt(currentFieldPosition);
if (currentCharacter == '{') {
braceLevel++;
} else if (currentCharacter == '}') {
braceLevel--;
}
// Append parameter content if we're still inside the braces
if (braceLevel > 0) {
cleanedField.append(currentCharacter);
}
currentFieldPosition++;
}
incommand = false;
escaped = false;
} else {
// End of the command (no parameters)
incommand = false;
escaped = false;
}
incommand = false;
escaped = false;
} else if (currentCharacter != '{' && currentCharacter != '}') {
// Append regular characters (non-command, non-braces)
cleanedField.append(currentCharacter);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,16 @@ void crossrefAndInAuthorNames() {
database), DEFAULT_UNWANTED_CHARACTERS));
}


@Test
void latexCommandsAreStrippedFromCitationKey() throws ParseException {
BibEntry entry = new BibEntry()
.withField(StandardField.TITLE, "Building \\mkbibquote{Community}");
String pattern = "[bibtexkey] - [fulltitle]";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I understand where your hacks with trim() and removal of - come from. However, you better use .withCitatoinKey("demo24") after new BibEntry() to set an citation key.

String generatedKey = generateKey(entry, pattern);
assertEquals("BuildingCommunity", generatedKey);
}

@Test
void andAuthorNames() throws ParseException {
String bibtexString = "@ARTICLE{whatevery, author={Mari D. Herland and Mona-Iren Hauge and Ingeborg M. Helgeland}}";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -444,7 +444,7 @@ void generateKeyRegExReplace() {
bibtexKeyPattern.setDefaultValue("[auth][year]");
entry.setField(StandardField.AUTHOR, "John Doe and Donald Smith and Will Wonder");
new CitationKeyGenerator(bibtexKeyPattern, database, preferences).generateAndSetKey(entry);
assertEquals(Optional.of("Doe3016"), entry.getCitationKey());
assertEquals(Optional.of("Doe2016"), entry.getCitationKey());
}

@Test
Expand Down
Loading