Package org.opengrok.indexer.analysis.eiffel

Class EiffelSymbolTokenizer

java.lang.Object
org.opengrok.indexer.analysis.JFlexStateStacker
org.opengrok.indexer.analysis.JFlexSymbolMatcher
org.opengrok.indexer.analysis.eiffel.EiffelSymbolTokenizer
All Implemented Interfaces:
JFlexJointLexer, JFlexLexer, JFlexStackingLexer, Resettable, ScanningSymbolMatcher, SymbolMatchedPublisher
public class EiffelSymbolTokenizer extends JFlexSymbolMatcher

  • Field Details

  • Constructor Details

    • EiffelSymbolTokenizer

      public EiffelSymbolTokenizer(Reader in)
      Creates a new scanner
      Parameters:
      in - the java.io.Reader to read input from.

  • Method Details

    • getYYCHAR

      public long getYYCHAR()
      Gets the yychar value.
      Returns:
      yychar

    • getYYEOF

      public int getYYEOF()
      Gets the YYEOF value.
      Specified by:
      getYYEOF in interface JFlexStackingLexer
      Specified by:
      getYYEOF in class JFlexStateStacker
      Returns:
      YYEOF

    • reset

      public void reset()
      Resets the Eiffel tracked state; Resets the instance to an initial state.
      Specified by:
      reset in interface Resettable

    • offer

      public void offer(String value) throws IOException
      Description copied from interface: JFlexJointLexer
      Passes non-symbolic fragment for processing.
      Parameters:
      value - the excised fragment
      Throws:
      IOException - if an error occurs while accepting

    • offerSymbol

      public boolean offerSymbol(String value, int captureOffset, boolean ignoreKwd) throws IOException
      Description copied from interface: JFlexJointLexer
      Passes a text fragment that is syntactically a symbol for processing.
      Parameters:
      value - the excised symbol
      captureOffset - the offset from yychar where value began
      ignoreKwd - a value indicating whether keywords should be ignored
      Returns:
      true if the value was not in keywords or if the ignoreKwd was true
      Throws:
      IOException - if an error occurs while accepting

    • skipSymbol

      public void skipSymbol()
      Description copied from interface: JFlexJointLexer
      Indicates that something unusual happened where normally a symbol would have been offered.

    • offerKeyword

      public void offerKeyword(String value) throws IOException
      Description copied from interface: JFlexJointLexer
      Passes a text fragment that is syntactically a keyword symbol for processing.
      Parameters:
      value - the excised symbol
      Throws:
      IOException - if an error occurs while accepting

    • startNewLine

      public void startNewLine() throws IOException
      Description copied from interface: JFlexJointLexer
      Indicates that the current line is ended.
      Throws:
      IOException - if an error occurs when handling the EOL

    • disjointSpan

      public void disjointSpan(String className) throws IOException
      Description copied from interface: JFlexJointLexer
      Indicates the closing of an open tag and the opening -- if className is non-null -- of a new one.
      Parameters:
      className - the class name for the new tag or null just to close an open tag.
      Throws:
      IOException - if an output error occurs

    • phLOC

      public void phLOC()
      Description copied from interface: JFlexJointLexer
      Indicates that eligible source code was encountered for physical lines-of-code count (physical LOC).

    • takeAllContent

      protected boolean takeAllContent()

    • returnOnSymbol

      protected boolean returnOnSymbol()

    • yyclose

      public final void yyclose() throws IOException
      Closes the input reader.
      Throws:
      IOException - if the reader could not be closed.

    • yyreset

      public final void yyreset(Reader reader)
      Resets the scanner to read from a new input stream.

      Does not close the old reader.

      All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to ZZ_INITIAL.

      Internal scan buffer is resized down to its initial length, if it has grown.

      Parameters:
      reader - The new input stream.

    • yyatEOF

      public final boolean yyatEOF()
      Returns whether the scanner has reached the end of the reader it reads from.
      Returns:
      whether the scanner has reached EOF.

    • yystate

      public final int yystate()
      Returns the current lexical state.
      Returns:
      the current lexical state.

    • yybegin

      public final void yybegin(int newState)
      Enters a new lexical state.
      Parameters:
      newState - the new lexical state

    • yytext

      public final String yytext()
      Returns the text matched by the current regular expression.
      Returns:
      the matched text.

    • yycharat

      public final char yycharat(int position)
      Returns the character at the given position from the matched text.

      It is equivalent to yytext().charAt(pos), but faster.

      Parameters:
      position - the position of the character to fetch. A value from 0 to yylength()-1.
      Returns:
      the character at position.

    • yylength

      public final int yylength()
      How many characters were matched.
      Returns:
      the length of the matched text region.

    • yypushback

      public void yypushback(int number)
      Pushes the specified amount of characters back into the input stream.

      They will be read again by then next call of the scanning method.

      Parameters:
      number - the number of characters to be read again. This number must not be greater than yylength().

    • yylex

      public int yylex() throws IOException
      Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
      Returns:
      the next token.
      Throws:
      IOException - if any I/O-Error occurs.

    • vop

      public void vop(String opener) throws IOException
      Begin a "Verbatim_string" according to the specified for the specified "Verbatim_string_opener", opener, determining the matching "Verbatim_string_closer" to be used with maybeEndVerbatim(java.lang.String).

      Everything between the first and last character of opener is the verbatim string α, and the last character's opposite becomes the initial character of the closer.

      Parameters:
      opener - a defined opener following Eiffel "8.29.10 Syntax: Manifest strings"
      Throws:
      IOException

    • maybeEndVerbatim

      public void maybeEndVerbatim(String capture) throws IOException
      Determine if the specified capture starts with the expected "Verbatim_string_closer" from vop(java.lang.String), and if so, pop the state, write the closer, and push back any excess; otherwise, write the first character of capture, and push back the rest.
      Parameters:
      capture - a defined, possible ender
      Throws:
      IOException - if output fails

    • chkLOC

      public void chkLOC()
      Calls JFlexJointLexer.phLOC() if the yystate is not SCOMMENT.