Building an IE8 Linter

$web-only$ To get started we need to add ress to our dependencies. This project is also going to need serde, serde_derive and toml because it will rely on a .toml file to make the list of unavailable tokens configurable.

[package]
name = "lint-ie8"
version = "0.1.0"
authors = ["Robert Masen <r@robertmasen.pizza>"]
edition = "2018"

[dependencies]
ress = "0.7"
serde = "1"
serde_derive = "1"
toml = "0.5"

Next we want to use the Scanner and Token from ress, we can do this by importing all the contents of the prelude.

#![allow(unused)]
fn main() {
use ress::prelude::*;
}

Since we are using a .toml file to provide the list of banned tokens, let's create a struct that will represent our configuration.

#![allow(unused)]
fn main() {
#[derive(Deserialize)]
struct BannedTokens {
    idents: Vec<String>,
    keywords: Vec<String>,
    puncts: Vec<String>,
    strings: Vec<String>,
}
}

The toml file we are going to use is pretty big so but if you want to see what it looks like you can check it out here. Essentially it is a list of identifiers, strings, punctuation, and keywords that would cause an error when trying to run in IE8.

To start we need to deserialize that file, we can do that with the std::fs::read_to_string and toml::from_str functions.

#![allow(unused)]
fn main() {
    let config_text = ::std::fs::read_to_string("banned_tokens.toml").expect("failed to read config");
    let banned: BannedTokens = from_str(&config_text).expect("Failed to deserialize banned tokens");
}

Now that we have a list of tokens that should not be included in our javascript, let's get the js text. It would be useful to be able to take a path argument or read the raw js from stdin. This function will check for an argument first and fallback to reading from stdin, it looks something like this.

#![allow(unused)]
fn main() {
fn get_js() -> Result<String, ::std::io::Error> {
    let mut cmd_args = args();
    let _ = cmd_args.next(); //discard bin name
    let js = if let Some(file_name) = cmd_args.next() {
        let js = read_to_string(file_name)?;
        js
    } else {
        let mut std_in = ::std::io::stdin();
        let mut ret = String::new();
        if std_in.is_terminal() {
            return Ok(ret)
        }
        std_in.read_to_string(&mut ret)?;
        ret
    };
    Ok(js)
}

}

we will call it like this.

#![allow(unused)]
fn main() {
    let js = match get_js() {
        Ok(js) => if js.len() == 0 {
            print_usage();
            std::process::exit(1);
        } else {
            js
        },
        Err(_) => {
            print_usage();
            std::process::exit(1);
        }
    };
    let finder = BannedFinder::new(&js, banned);
}

We want to handle the failure when attempting to get the js, so we will match on the call to get_js. If everything went well we need to check if the text is an empty string, this means no argument was provided but the program was not pipped any text. In either of these failure cases we want to print a nice message about how the command should have been written and then exit with a non-zero status code. print_usage is a pretty simple function that will just print to stdout the two ways to use the program.

#![allow(unused)]
fn main() {
fn print_usage() {
    println!("banned_tokens <infile>
cat <path/to/file> | banned_tokens");
}
}

With that out of the way, we now can get into how we are going to solve the actual problem of finding these tokens in a javascript file. There are many ways to make this work but for this example we are going to wrap the Scanner in another struct that implements Iterator. First here is what that struct is going to look like.

#![allow(unused)]
fn main() {
struct BannedFinder<'a> {
    scanner: Scanner<'a>,
    banned: BannedTokens,
}

}

Before we get into the impl Iterator we should go over an Error implementation that we are going to use. It is relatively straight forward, the actual struct is going to be a tuple struct with three items. The first item is going to be a message that will include the token and type, the second and third are going to be the column/row of the banned token. We need to implement display (Error requires it) which will just create a nice error message for us.

#![allow(unused)]
fn main() {
#[derive(Debug)]
pub struct BannedError(String, usize, usize);

impl ::std::error::Error for BannedError {

}

impl ::std::fmt::Display for BannedError {
    fn fmt(&self, f: &mut ::std::fmt::Formatter) -> ::std::fmt::Result {
        write!(f, "Banned {} found at {}:{}", self.0, self.1, self.2)
    }
}
}

Now we can add a method to BannedFinder that will take an index and return the row/column pair.

Ok, now for the exciting part; we are going to impl Iterator for BannedFinder which will look like this.

#![allow(unused)]
fn main() {
impl<'a> Iterator for BannedFinder<'a> {
    type Item = Result<(), BannedError>;
    fn next(&mut self) -> Option<Self::Item> {
        if let Some(item) = self.scanner.next() {
            match item {
                Ok(item) => {
                    Some(match &item.token {
                        Token::Ident(ref id) => {
                            let id = id.to_string();
                            if self.banned.idents.contains(&id) {
                                Err(BannedError(format!("identifier {}", id), item.location.start.line, item.location.start.column))
                            } else {
                                Ok(())
                            }
                        },
                        Token::Keyword(ref key) => {
                            if self.banned.keywords.contains(&key.to_string()) {
                                Err(BannedError(format!("keyword {}", key.to_string()), item.location.start.line, item.location.start.column))
                            } else {
                                Ok(())
                            }
                        },
                        Token::Punct(ref punct) => {
                            if self.banned.puncts.contains(&punct.to_string()) {
                                Err(BannedError(format!("punct {}", punct.to_string()), item.location.start.line, item.location.start.column))
                            } else {
                                Ok(())
                            }
                        },
                        Token::String(ref lit) => {
                            match lit {
                                StringLit::Double(inner)
                                | StringLit::Single(inner) => {
                                    if self.banned.strings.contains(&inner.to_string()) {
                                        Err(BannedError(format!("string {}", lit.to_string()), item.location.start.line, item.location.start.column))
                                    } else {
                                        Ok(())
                                    }
                                }
                            }
                        },
                        _ => Ok(()),
                    })
                },
                Err(_) => {
                    None
                }
            }
        } else {
            None
        }
    }
}

}

First we need to define what the Item for our Iterator is. It is going to be a Result<(), BannedError>, this will allow the caller to check if an item passed inspection. Now we can add the fn next(&mut self) -> Option<Self::Item> definition. Inside that we first want to make sure that the Scanner isn't returning None, if it is we can just return None. If the scanner returns and Result<Item, Error> we first need to check that it is Ok, in this example we are just going to ignore the Err case. Once we have an actual Item we want to check what kind of token it is, we can do that by matching on &item.token. We only care if the token is a Keyword, Ident, Punct or String, other wise we can say that the token passed. For each of these tokens we are going to check if the actual text is included in any of the Vec<String> properties of self.banned, if it is included we return a BannedError where the first property is a message containing the name of the token type and the text that token represents.

Now that we have all of the underlying infrastructure setup, let's use the BannedFinder in our main.

#![allow(unused)]
fn main() {
    let finder = BannedFinder::new(&js, banned);
    for item in finder {
        match item {
            Ok(_) => (),
            Err(msg) => println!("{}", msg),
        }
    }
}

That is pretty much it. If you wanted to see the full project you can find it in the lint-ie8 folder of this book's github repository.

$web-only-end$ $slides-only$

Demo

$slides-only-end$