Package com.blackrook.expression.struct

Class Lexer.Kernel

  • Enclosing class:
    Lexer
    public static class Lexer.Kernel
extends Object
    This is a info kernel that tells a Lexer how to interpret certain characters and identifiers.
    Author:
    Matthew Tropiano

    • Field Detail

      • TYPE_END_OF_LEXER

        public static final int TYPE_END_OF_LEXER
        Reserved token type: End of lexer.
        See Also:
        Constant Field Values

      • TYPE_END_OF_STREAM

        public static final int TYPE_END_OF_STREAM
        Reserved token type: End of stream.
        See Also:
        Constant Field Values

      • TYPE_NUMBER

        public static final int TYPE_NUMBER
        Reserved token type: Number.
        See Also:
        Constant Field Values

      • TYPE_DELIM_SPACE

        public static final int TYPE_DELIM_SPACE
        Reserved token type: Space.
        See Also:
        Constant Field Values

      • TYPE_DELIM_TAB

        public static final int TYPE_DELIM_TAB
        Reserved token type: Tab.
        See Also:
        Constant Field Values

      • TYPE_DELIM_NEWLINE

        public static final int TYPE_DELIM_NEWLINE
        Reserved token type: New line character.
        See Also:
        Constant Field Values

      • TYPE_DELIM_OPEN_COMMENT

        public static final int TYPE_DELIM_OPEN_COMMENT
        Reserved token type: Open comment.
        See Also:
        Constant Field Values

      • TYPE_DELIM_CLOSE_COMMENT

        public static final int TYPE_DELIM_CLOSE_COMMENT
        Reserved token type: Close comment.
        See Also:
        Constant Field Values

      • TYPE_DELIM_LINE_COMMENT

        public static final int TYPE_DELIM_LINE_COMMENT
        Reserved token type: Line comment.
        See Also:
        Constant Field Values

      • TYPE_IDENTIFIER

        public static final int TYPE_IDENTIFIER
        Reserved token type: Identifier.
        See Also:
        Constant Field Values

      • TYPE_UNKNOWN

        public static final int TYPE_UNKNOWN
        Reserved token type: Unknown token.
        See Also:
        Constant Field Values

      • TYPE_ILLEGAL

        public static final int TYPE_ILLEGAL
        Reserved token type: Illegal token.
        See Also:
        Constant Field Values

      • TYPE_COMMENT

        public static final int TYPE_COMMENT
        Reserved token type: Comment.
        See Also:
        Constant Field Values

      • TYPE_LINE_COMMENT

        public static final int TYPE_LINE_COMMENT
        Reserved token type: Line Comment.
        See Also:
        Constant Field Values

      • TYPE_STRING

        public static final int TYPE_STRING
        Reserved token type: String.
        See Also:
        Constant Field Values

      • TYPE_RAWSTRING

        public static final int TYPE_RAWSTRING
        Reserved token type: Raw String (never returned).
        See Also:
        Constant Field Values

      • TYPE_DELIMITER

        public static final int TYPE_DELIMITER
        Reserved token type: Delimiter (never returned).
        See Also:
        Constant Field Values

      • TYPE_POINT

        public static final int TYPE_POINT
        Reserved token type: Point state (never returned).
        See Also:
        Constant Field Values

      • TYPE_FLOAT

        public static final int TYPE_FLOAT
        Reserved token type: Floating point state (never returned).
        See Also:
        Constant Field Values

      • TYPE_DELIM_COMMENT

        public static final int TYPE_DELIM_COMMENT
        Reserved token type: Delimiter Comment (never returned).
        See Also:
        Constant Field Values

      • TYPE_HEX_INTEGER0

        public static final int TYPE_HEX_INTEGER0
        Reserved token type: hexadecimal integer (never returned).
        See Also:
        Constant Field Values

      • TYPE_HEX_INTEGER1

        public static final int TYPE_HEX_INTEGER1
        Reserved token type: hexadecimal integer (never returned).
        See Also:
        Constant Field Values

      • TYPE_HEX_INTEGER

        public static final int TYPE_HEX_INTEGER
        Reserved token type: hexadecimal integer (never returned).
        See Also:
        Constant Field Values

      • TYPE_EXPONENT

        public static final int TYPE_EXPONENT
        Reserved token type: Exponent state (never returned).
        See Also:
        Constant Field Values

      • TYPE_EXPONENT_POWER

        public static final int TYPE_EXPONENT_POWER
        Reserved token type: Exponent power state (never returned).
        See Also:
        Constant Field Values

    • Constructor Detail

      • Kernel

        public Kernel()
        Creates a new, blank LexerKernel with default settings.

    • Method Detail

      • addDelimiter

        public void addDelimiter​(String delimiter,
                         int type)
        Adds a delimiter to this lexer.
        Parameters:
        delimiter - the delimiter lexeme.
        type - the type id.
        Throws:
        IllegalArgumentException - if type is < 0 or delimiter is null or empty.

      • addStringDelimiter

        public void addStringDelimiter​(char delimiterStart,
                               char delimiterEnd)
        Adds a string delimiter to this lexer along with its ending character.
        Parameters:
        delimiterStart - the starting delimiter.
        delimiterEnd - the ending delimiter.

      • addRawStringDelimiter

        public void addRawStringDelimiter​(char delimiterStart,
                                  char delimiterEnd)
        Adds a raw string delimiter to this lexer along with its ending character.
        Parameters:
        delimiterStart - the starting delimiter.
        delimiterEnd - the ending delimiter.

      • addCommentStartDelimiter

        public void addCommentStartDelimiter​(String delimiter,
                                     int type)
        Adds a comment-starting delimiter to this lexer.
        Parameters:
        delimiter - the delimiter lexeme.
        type - the type id.
        Throws:
        IllegalArgumentException - if type is < 0 or delimiter is null or empty.

      • addCommentEndDelimiter

        public void addCommentEndDelimiter​(String delimiter,
                                   int type)
        Adds a comment-ending delimiter to this lexer.
        Parameters:
        delimiter - the delimiter lexeme.
        type - the type id.
        Throws:
        IllegalArgumentException - if type is < 0 or delimiter is null or empty.

      • addCommentLineDelimiter

        public void addCommentLineDelimiter​(String delimiter,
                                    int type)
        Adds a line comment delimiter to this lexer.
        Parameters:
        delimiter - the delimiter lexeme.
        type - the type id.
        Throws:
        IllegalArgumentException - if type is < 0 or delimiter is null or empty.

      • addKeyword

        public void addKeyword​(String keyword,
                       int type)
        Adds a keyword to the Lexer, case-sensitive. When this identifier is read in, its token type is specified type.
        Parameters:
        keyword - the keyword identifier.
        type - the type id.

      • addCaseInsensitiveKeyword

        public void addCaseInsensitiveKeyword​(String keyword,
                                      int type)
        Adds a keyword to the Lexer, case-insensitive. When this identifier is read in, its token type is specified type.
        Parameters:
        keyword - the keyword identifier.
        type - the type id.

      • willEmitSpaces

        public boolean willEmitSpaces()
        Checks if this lexer emits space tokens.
        Returns:
        true if so, false if not.

      • setEmitSpaces

        public void setEmitSpaces​(boolean includeSpaces)
        Sets if this lexer emits space tokens?
        Parameters:
        includeSpaces - true if so, false if not.

      • willEmitTabs

        public boolean willEmitTabs()
        Checks if this lexer emits tab tokens.
        Returns:
        true if so, false if not.

      • setEmitTabs

        public void setEmitTabs​(boolean includeTabs)
        Sets if this lexer emits tab tokens.
        Parameters:
        includeTabs - true if so, false if not.

      • willEmitNewlines

        public boolean willEmitNewlines()
        Checks if this lexer emits newline tokens.
        Returns:
        true if so, false if not.

      • setEmitNewlines

        public void setEmitNewlines​(boolean includeNewlines)
        Sets if this lexer emits newline tokens.
        Parameters:
        includeNewlines - true if so, false if not.

      • willEmitStreamBreak

        public boolean willEmitStreamBreak()
        Checks if this lexer emits stream break tokens.
        Returns:
        true if so, false if not.

      • setEmitStreamBreak

        public void setEmitStreamBreak​(boolean emitStreamBreak)
        Sets if this lexer emits stream break tokens.
        Parameters:
        emitStreamBreak - true if so, false if not.

      • willEmitComments

        public boolean willEmitComments()
        Checks if this lexer emits comment tokens.
        Returns:
        true if so, false if not.

      • setEmitComments

        public void setEmitComments​(boolean emitComments)
        Sets if this lexer emits comment tokens.
        Parameters:
        emitComments - true if so, false if not.

      • setDecimalSeparator

        public void setDecimalSeparator​(char c)
        Sets the current decimal separator character. By default, this is the current locale's decimal separator character.
        Parameters:
        c - the character to set.

      • getDecimalSeparator

        public char getDecimalSeparator()
        Gets the current decimal separator character. By default, this is the current locale's decimal separator character.
        Returns:
        the separator character.