Package org.opengrok.indexer.analysis.r

Class RSymbolTokenizer

java.lang.Object
org.opengrok.indexer.analysis.JFlexStateStacker
org.opengrok.indexer.analysis.JFlexSymbolMatcher
org.opengrok.indexer.analysis.r.RLexer
org.opengrok.indexer.analysis.r.RSymbolTokenizer
All Implemented Interfaces:
JFlexJointLexer, JFlexLexer, JFlexStackingLexer, Resettable, ScanningSymbolMatcher, SymbolMatchedPublisher
public class RSymbolTokenizer extends RLexer

  • Field Details

  • Constructor Details

    • RSymbolTokenizer

      public RSymbolTokenizer(Reader in)
      Creates a new scanner
      Parameters:
      in - the java.io.Reader to read input from.

  • Method Details

    • getYYCHAR

      public long getYYCHAR()
      Gets the yychar value.
      Returns:
      yychar

    • getYYEOF

      public int getYYEOF()
      Gets the YYEOF value.
      Specified by:
      getYYEOF in interface JFlexStackingLexer
      Specified by:
      getYYEOF in class JFlexStateStacker
      Returns:
      YYEOF

    • reset

      public void reset()
      Resets the R tracked state; Resets the instance using JFlexStateStacker.clearStack(), and sets line number to one.
      Specified by:
      reset in interface Resettable
      Overrides:
      reset in class JFlexStateStacker

    • offer

      public void offer(String value) throws IOException
      Does nothing.
      Parameters:
      value - the excised fragment
      Throws:
      IOException - if an error occurs while accepting

    • offerSymbol

      public boolean offerSymbol(String value, int captureOffset, boolean ignoreKwd) throws IOException
      Description copied from interface: JFlexJointLexer
      Passes a text fragment that is syntactically a symbol for processing.
      Parameters:
      value - the excised symbol
      captureOffset - the offset from yychar where value began
      ignoreKwd - a value indicating whether keywords should be ignored
      Returns:
      true if the value was not in keywords or if the ignoreKwd was true
      Throws:
      IOException - if an error occurs while accepting

    • skipSymbol

      public void skipSymbol()
      Just forget any last-matched symbol.

    • offerKeyword

      public void offerKeyword(String value) throws IOException
      Just forget any last-matched symbol.
      Parameters:
      value - the excised symbol
      Throws:
      IOException - if an error occurs while accepting

    • startNewLine

      public void startNewLine() throws IOException
      Does nothing.
      Throws:
      IOException - if an error occurs when handling the EOL

    • disjointSpan

      public void disjointSpan(String className) throws IOException
      Does nothing.
      Parameters:
      className - the class name for the new tag or null just to close an open tag.
      Throws:
      IOException - if an output error occurs

    • phLOC

      public void phLOC()
      Does nothing.

    • takeAllContent

      protected boolean takeAllContent()
      Gets the value false.

    • returnOnSymbol

      protected boolean returnOnSymbol()
      Gets a value indicating if a symbol was just matched.

    • SCOMMENT

      public int SCOMMENT()
      Gets the constant value created by JFlex to represent SCOMMENT.
      Specified by:
      SCOMMENT in class RLexer

    • yyclose

      public final void yyclose() throws IOException
      Closes the input reader.
      Throws:
      IOException - if the reader could not be closed.

    • yyreset

      public final void yyreset(Reader reader)
      Resets the scanner to read from a new input stream.

      Does not close the old reader.

      All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to ZZ_INITIAL.

      Internal scan buffer is resized down to its initial length, if it has grown.

      Parameters:
      reader - The new input stream.

    • yyatEOF

      public final boolean yyatEOF()
      Returns whether the scanner has reached the end of the reader it reads from.
      Returns:
      whether the scanner has reached EOF.

    • yystate

      public final int yystate()
      Returns the current lexical state.
      Returns:
      the current lexical state.

    • yybegin

      public final void yybegin(int newState)
      Enters a new lexical state.
      Parameters:
      newState - the new lexical state

    • yytext

      public final String yytext()
      Returns the text matched by the current regular expression.
      Returns:
      the matched text.

    • yycharat

      public final char yycharat(int position)
      Returns the character at the given position from the matched text.

      It is equivalent to yytext().charAt(pos), but faster.

      Parameters:
      position - the position of the character to fetch. A value from 0 to yylength()-1.
      Returns:
      the character at position.

    • yylength

      public final int yylength()
      How many characters were matched.
      Returns:
      the length of the matched text region.

    • yypushback

      public void yypushback(int number)
      Pushes the specified amount of characters back into the input stream.

      They will be read again by then next call of the scanning method.

      Parameters:
      number - the number of characters to be read again. This number must not be greater than yylength().

    • yylex

      public int yylex() throws IOException
      Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
      Returns:
      the next token.
      Throws:
      IOException - if any I/O-Error occurs.