Skip to content

first stab on fixing #1170 #1346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jan 27, 2017
Merged

first stab on fixing #1170 #1346

merged 3 commits into from
Jan 27, 2017

Conversation

@tarzanek
Copy link
Contributor

@tarzanek tarzanek commented Jan 26, 2017 •
edited by tulinkry
Loading

@tarzanek
Copy link
Contributor Author

tarzanek commented Jan 26, 2017

basically the only two things missing are:

  1. pom.xml integration
  2. position incrementor as in https://issues.apache.org/jira/secure/attachment/12663628/LUCENE-5897.patch - this needs to be tested, junit do some of tests, so let's see if they pass, I will run also helius farm job to get full checks
@vladak
Copy link
Member

vladak commented Jan 26, 2017

Couple of comments:

  • turn this on for javascript analyzer
  • let xref to be generated
  • fix the Maven somehow
  • produce some sort of message when the file is skipped
@tarzanek tarzanek self-assigned this Jan 26, 2017
@tarzanek tarzanek added this to the 0.13 milestone Jan 26, 2017
tulinkry
tulinkry suggested changes Jan 26, 2017
View reviewed changes
src/org/opensolaris/opengrok/analysis/plain/PlainSymbolTokenizer.lex
import org.opensolaris.opengrok.analysis.JFlexTokenizer;
%%
%public
%class PlainSymbolTokenizer
Copy link
Contributor

@tulinkry tulinkry Jan 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright
src/org/opensolaris/opengrok/analysis/plain/PlainFullTokenizer.lex
%%

%public
%class PlainFullTokenizer
Copy link
Contributor

@tulinkry tulinkry Jan 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright
src/org/opensolaris/opengrok/analysis/javascript/JavaScriptSymbolTokenizer.lex

%%
%public
%class JavaScriptSymbolTokenizer
Copy link
Contributor

@tulinkry tulinkry Jan 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright
src/org/opensolaris/opengrok/analysis/java/JavaSymbolTokenizer.lex
%class JavaSymbolTokenizer
%extends JFlexTokenizer
%init{
super(in);
Copy link
Contributor

@tulinkry tulinkry Jan 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyright
opengrok-indexer/pom.xml
</execution>
</executions>
</plugin>
<!-- TODO add the same fix as is in build.xml to patch jflex generated files to stop increasing buffer beyond token size that lucene accepts
Copy link
Contributor

@tulinkry tulinkry Jan 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe copyright?
@tarzanek
Copy link
Contributor Author

tarzanek commented Jan 27, 2017

@tulinkry , copyright fixed ... will you have time to add the regexps to pom.xml ?
@tarzanek
Copy link
Contributor Author

tarzanek commented Jan 27, 2017

let's merge this, let the fun begin :-D
@tarzanek tarzanek merged commit e4df909 into oracle:master Jan 27, 2017
@tarzanek
Copy link
Contributor Author

tarzanek commented Jan 27, 2017

fwiw -
fix the Maven somehow - let's sort this out in next pull req
produce some sort of message when the file is skipped - file won't be skipped, just the buggy token, which I think is safe to ignore, I am still afraid the index size and search speed will be impacted by this, since we might get 1MB doc fields thanks to this ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants

@tarzanek @vladak @tulinkry