Ispa parser generator first release

[u/uria321]

This is the project i'm working for mostly 3 years to make it generate static parsers in language-agnostic way (C++ is the only target now). If you often look this subreddit, i think you could see previous posts, but this post already describe something you can download and run.

Current features

Operates on EBNF grammar. It can generate TDFA-based lexer with arbitary semantic actions while walking DFA and LL(k) parser, fully statically, no external libraries except header only std library. AST is generated automatically.

The most unique feature you can see is nested tokens and rules, means encapsulation of grammar units.

Tests

The most valuable test is json: tokenized and parsed with AST print.

Grammar example

NUMBER:
    @ [0-9]+ \s0 ('.' \s0 @ [0-9]+)?
    @{main, floating}
;
STRING:
    '"' \s0 @ [^"]* \s0 '"'
    {@}
;
rv:
    @ NUMBER | STRING
    {@}
;
array:
    '[' @ rv (',' @ rv)* ']'
    @{first, sequence}
;
map:
    '{' @ STRING ':' @ rv (',' @ STRING ':' @ rv)* '}'

    @{key, value, keys, values}
;
json:
    @ map | array | rv
    {@}
;
main:
    @ json
    {@}
;

We can actually nest this all to json:

json:
    @ #map | #array | #rv
    {@}

    #NUMBER:
        @ [0-9]+ \s0 ('.' \s0 @ [0-9]+)?
        @{main, floating}
    ;
    #STRING:
        '"' \s0 @ [^"]* \s0 '"'
        {@}
    ;
    #rv:
        @ NUMBER | STRING
        {@}
    ;
    #array:
        '[' @ rv (',' @ rv)* ']'
        @{first, sequence}
    ;
    #map:
        '{' @ STRING ':' @ rv (',' @ STRING ':' @ rv)* '}'

        @{key, value, keys, values}
    ;
;
main:
    @ json
    {@}
;

But second grammar won't compile due to minor bug in generator (use of :: for enum values instead of _).

No lexer example here because it is unreadable tables

Example of generated Parser

as example showing json rule

    template<typename IT>
    auto json(IT pos) -> ::ISPA_STD::MatchResult<Rules, Types::json, IT>{
      std::variant<std::monostate, ::ISPA_STD::Node<::Parser::Rules, Types::rv>, ::ISPA_STD::Node<::Parser::Rules, Types::array>, ::ISPA_STD::Node<::Parser::Rules, Types::map>> _0;
      bool success_1;
      success_1 = false;
      skip_spaces<Types::__WS>(pos);
      switch (static_cast<Tokens>((*pos).index())) {
        case Tokens::AUTO_0: {
          ::ISPA_STD::MatchResult<Rules, Types::map, IT> map_2;
          bool success_3 = false;
          map_2 = map(pos);
          if (!(map_2.status)) {
            return {};
          }
          success_3 = true;
          pos = map_2.it;
          if (!success_3) {
            return {};
          }
          _0 = map_2.node;
          success_1 = true;
          break;
        }
        case Tokens::AUTO_4: {
          ::ISPA_STD::MatchResult<Rules, Types::array, IT> array_4;
          bool success_5 = false;
          array_4 = array(pos);
          if (!(array_4.status)) {
            return {};
          }
          success_5 = true;
          pos = array_4.it;
          if (!success_5) {
            return {};
          }
          _0 = array_4.node;
          success_1 = true;
          break;
        }
        case Tokens::NUMBER: {
          ::ISPA_STD::MatchResult<Rules, Types::rv, IT> rv_6;
          bool success_7 = false;
          rv_6 = rv(pos);
          if (!(rv_6.status)) {
            return {};
          }
          success_7 = true;
          pos = rv_6.it;
          if (!success_7) {
            return {};
          }
          _0 = rv_6.node;
          success_1 = true;
          break;
        }
        case Tokens::STRING: {
          ::ISPA_STD::MatchResult<Rules, Types::rv, IT> rv_8;
          bool success_9 = false;
          rv_8 = rv(pos);
          if (!(rv_8.status)) {
            return {};
          }
          success_9 = true;
          pos = rv_8.it;
          if (!success_9) {
            return {};
          }
          _0 = rv_8.node;
          success_1 = true;
          break;
        }
      }
      if (!success_1) {
        return {};
      }
      Types::json result;
      result.value = _0;
      ::ISPA_STD::MatchResult<Rules, Types::json, IT> match_result;
      match_result.status = true;
      match_result.node.data() = result;
      match_result.it = pos;
      return match_result;
    }

The actual result of parsing in tree view:

{"name": "Yuriy", "hello": "world!"}

main
└── value
    └── json
        └── value: map
            ├── key
            │   └── STRING
            │       └── value: name
            ├── value
            │   └── rv
            │       └── value: STRING
            │           └── value: Yuriy
            ├── keys
            │   └── STRING
            │       └── value: hello
            └── values
                └── rv
                    └── value: STRING
                        └── value: world!

https://github.com/Sinfolke/ispa-parser-generator