RESS Scanners

In the initial implementation of the ress scanner, it was more important to get something working correctly than to have something blazing fast. To that end, the original Scanner performs a significant amount of memory allocation, which slows everything down quite a bit. To improve upon that ress offers a section option the RefScanner, which is a bit unfortunately named as it doesn't actually use any references. The RefScanner provides almost the same information as the Scanner but it does so without making any copies from the original javascript string, it the has the option to request the String for any Item giving the control to the user. Here is an example of the two approaches.

Example JS

function things() {
    return [1,2,3,4];
}

Example Rust

use ress::{
    Scanner
};
fn main() {
    let js = include_str!("../example.js");
    let scanner = Scanner::new(js);
    for (i, item) in scanner.enumerate() {
        let item = item.unwrap();
        let prefix = if i < 10 {
            format!(" {}", i)
        } else {
            format!("{}", i)
        };
        println!("{} token: {:?}", prefix, item.token);
    }
}

#[cfg(test)]
mod test {
    use ress::*;
    #[test]
    fn chapter_1_1() {
        let js = "var i = 0;";
        let scanner = Scanner::new(js);
        for token in scanner {
            println!("{:#?}", token.unwrap());
        }
    }
    use ressa::Parser;
    #[test]
    fn ressa_ex1() {
        static JS: &str = "
function Thing(stuff) {
    this.stuff = stuff;
}
";
        let parser = Parser::new(JS).expect("Failed to create parser");
        for part in parser {
            let part = part.expect("Failed to parse part");
            println!("{:#?}", part);
        }
    }
}

Output

running 1 test
Decl(
    Func(
        Func {
            id: Some(
                Ident {
                    name: "Thing",
                },
            ),
            params: [
                Pat(
                    Ident(
                        Ident {
                            name: "stuff",
                        },
                    ),
                ),
            ],
            body: FuncBody(
                [
                    Stmt(
                        Expr(
                            Assign(
                                AssignExpr {
                                    operator: Equal,
                                    left: Expr(
                                        Member(
                                            MemberExpr {
                                                object: This,
                                                property: Ident(
                                                    Ident {
                                                        name: "stuff",
                                                    },
                                                ),
                                                computed: false,
                                            },
                                        ),
                                    ),
                                    right: Ident(
                                        Ident {
                                            name: "stuff",
                                        },
                                    ),
                                },
                            ),
                        ),
                    ),
                ],
            ),
            generator: false,
            is_async: false,
        },
    ),
)
test test::ressa_ex1 ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out

Let's look at token 7, the original token is Token::Numeric(Number(String::From("1"))) while the ref token is Token::Numeric(Number::Dec), both give similar information but the ref token doesn't allocate a new string for the text being represented, instead just informing the user that it is a decimal number. If you wanted to know what that string was, you could use the RefScanner::string_for method by passing it RefItem.span, this will return an Option<String> and so long as your span doesn't overflow the length of the js provided, it will have the value you are looking for.