ParseJS. An easy function I made to tokenize things.

ParseJS on GitHub

Want to make a math evaluater? A programming language mockup? A document format? A file reader?
If you want to make any of those, my small one-function library is for you! 😀

This is the function signature:

function parse_string(…


This content originally appeared on DEV Community and was authored by Calin Baenen

ParseJS on GitHub

Want to make a math evaluater? A programming language mockup? A document format? A file reader?
If you want to make any of those, my small one-function library is for you! :D

This is the function signature:

function parse_string(
    str:string,
    toks:Array<string>
):Array<string|symbol>;

How does it work?
It takes a string to scan and a list of tokens. If a token is found; a symbol (Symbol.for( tok )) is inserted, otherwise the character in the string being scanned is inserted -- and the final result is returned.

Here is an example of the function in action!
"Sonic", "Tails", and "Knuckles" are all recognized as tokens, but "Amy" isn't.

The documentation looks like this:

Parses a string and returns a list consisting of one of
length-1-`string` (`char`) or `symbol`, representing the
tokens the scanner finds.
Every time a token is successfully found, a symbol
representing the token is inserted, otherwise
the current character is used instead.


This content originally appeared on DEV Community and was authored by Calin Baenen


Print Share Comment Cite Upload Translate Updates
APA

Calin Baenen | Sciencx (2021-12-11T09:45:45+00:00) ParseJS. An easy function I made to tokenize things.. Retrieved from https://www.scien.cx/2021/12/11/parsejs-an-easy-function-i-made-to-tokenize-things/

MLA
" » ParseJS. An easy function I made to tokenize things.." Calin Baenen | Sciencx - Saturday December 11, 2021, https://www.scien.cx/2021/12/11/parsejs-an-easy-function-i-made-to-tokenize-things/
HARVARD
Calin Baenen | Sciencx Saturday December 11, 2021 » ParseJS. An easy function I made to tokenize things.., viewed ,<https://www.scien.cx/2021/12/11/parsejs-an-easy-function-i-made-to-tokenize-things/>
VANCOUVER
Calin Baenen | Sciencx - » ParseJS. An easy function I made to tokenize things.. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2021/12/11/parsejs-an-easy-function-i-made-to-tokenize-things/
CHICAGO
" » ParseJS. An easy function I made to tokenize things.." Calin Baenen | Sciencx - Accessed . https://www.scien.cx/2021/12/11/parsejs-an-easy-function-i-made-to-tokenize-things/
IEEE
" » ParseJS. An easy function I made to tokenize things.." Calin Baenen | Sciencx [Online]. Available: https://www.scien.cx/2021/12/11/parsejs-an-easy-function-i-made-to-tokenize-things/. [Accessed: ]
rf:citation
» ParseJS. An easy function I made to tokenize things. | Calin Baenen | Sciencx | https://www.scien.cx/2021/12/11/parsejs-an-easy-function-i-made-to-tokenize-things/ |

Please log in to upload a file.




There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.