Why every frontend developer should know ASTs
What is AST?
In computer science, an abstract syntax tree (AST), or just syntax tree, is a tree representation of the abstract syntactic structure of the text (often source code) written in a formal language. Each node of the tree denotes a construct occurring in the text.
ASTs are being used in interpreters and compilers, for static code analysis. For instance, you might want to create a tool that will decrease duplicate of code or create a transpiler, which will transform the code from Python to Javascript. Before the code transforms to AST it looks like plain text, after parsing it becomes a tree structure, which contains the same information as the source code. As a result, we can transform the source code to AST and vice versa.
Let's take a look on example:
function sum(a, b) {
if (a + b >= 100) return 100;
return a + b;
}Parser will create abstract syntax tree, which is schematically represented below:
At the top of the tree is a node representing the full program. This node has a body, containing all the top-level statements you wrote down. Going deeper, every statement has other sub-children representing code that is a part of that node, and so on and so on.
It's simplified representation of our code, to look into a real AST tree you can put it to AST explorer and choose the parser you need. I recommended you to play around with that a bit, if this is your first time seeing the AST.
AST for a frontend developer
Plugins creation
A lot of tools in frontend use AST transformations under the hood, like Babel, Typescript, Eslint, Prettier, Remark, Rehype, Minify, Uglify and etc. In most cases with Babel and Eslint, you can apply already existing plugin to Babel or Eslint, but sometimes you might need to create your own plugin and that's where the knowledge of ASTs can be useful!
For example, in your company, you have some code standards and you want to prevent newcomers and developers from mistakes, and enforce the rules specific to your codebase. This can be achieved by creating your own eslint plugin.
Let's say we have to support multidirectional layouts. Sometimes developrs can forget about that and write code like this:
import styled from "styled-components";
const StyledWrapper = styled.div`
margin-right: 15px;
`;Our eslint plugin can warn developer and remind him to use helper function, to rewrite the code to:
import styled from "styled-components";
import { right } from "@utils/rtl";
const StyledWrapper = styled.div`
margin-${right}: 15px;
`;In that case our eslint rule code can look something like this:
const useRtl = {
meta: {
type: "problem",
docs: {
description: "Checks if the static literal values require RTL function",
category: "Possible Errors",
recommended: true,
},
},
create: ctx => {
const rules = ["right", "left", "padding-left", "padding-right", "margin-left", "margin-right"];
return {
TemplateElement(path) {
const { node } = path;
// helper function which checks, that current template node is a styled-component
if (isStyledTagName(node)) {
if (node.value && node.value.cooked) {
const properties = node.value.cooked
.replace(/\n/gm, "")
.trim()
.split(";")
.map(v => v.trim())
.filter(Boolean)
.map(p => p.split(":"));
properties.forEach(([prop, value]) => {
if (rules.includes(prop) && value) {
context.report({
node,
message:
"Values for right and left edges are different. Use rtl utility function",
});
}
});
}
}
},
};
},
};It's a simple example. The interesting part here starts from TemplateElement(path). It's a node type on which ESLint will call visitor function. If you would like to go deeper with AST trees, for better understanding, you can read about visitor pattern which
is useful to visit complex object structures and usually being used to traverse AST trees.
In the example above babel-eslint parser was used and we did not remove or modified the current node,
we just explored it and called eslint reporter.
Pretty simple, right?
One of the most used parsers is @babel/parser, formerly known as babylon. It comes with everything you need to traverse and transform AST.
You can also use the @babel/types(Babel Types is a Lodash-esque utility library for AST nodes), which can help you to add checks for nodes during the traverse process or to create new nodes.
import { types as t, parse as parser } from "@babel/core";
...
path.node.declaration.properties.forEach(prop => {
// check if node is ObjectProperty
if (t.isObjectProperty(prop)) {
// check if key is Identifier
if (t.isIdentifier(prop.key) && prop.key.name === name) {
...
}
}
...
// or create new nodes like that
...
// will generate a new attribute with string value
const generateAttribute = (name, value) =>
t.jsxAttribute(t.jsxIdentifier(name), t.stringLiteral(value));
...import { traverse } from "@babel/core";
// you can remove the node from AST tree like that:
traverse(ast, {
// will remove typescript type annotations
TSTypeAnnotation: path => {
path.remove();
},
// will remove typescript type assertions
TSTypeParameterInstantiation: path => {
path.remove();
},
});
});
Interactive playground
Another example is where I found AST transformations to be useful.
We had to add playgrounds for our components in our design system documentation. We decided to use
react-live, the library, which helps to create an interactive playground, it's easy to use, and has a good API. My friend and colleague
made a great design for the whole new documentation including a playground, the design proposal also included a tab called playground, where it should be possible
to change the component's props values with toggles, selects, text fields, something like storybook knobs feature.
It requires us to pass code property with a string value.
So it's the perfect case for AST transformations because parser will receive it as a string, we can do some transformations based on our knobs choices and generate the code to react-live.
Something like that:
// we use @babel/standalone here, because it's possible to use in browser
import { transform as babelTransform } from "@babel/standalone";
import { LiveProvider, LiveEditor, LiveError, LivePreview } from "react-live";
import React from "react";
import Button from "@kiwicom/orbit-components/lib/Button";
import Switch from "@kiwicom/orbit-components/lib/Switch";
import Stack from "@kiwicom/orbit-components/lib/Stack";
import styled from "styled-components";
const transform = (code, attrs) => {
const result = babelTransform(code, {
filename: "",
sourceType: "module",
presets: ["react"],
plugins: [
({ types: t }) => {
return {
visitor: {
JSXAttribute: path => {
const { name: currentName } = path.node;
const newValue = attrs[currentName.name];
if (typeof newValue !== "undefined") {
// creates node with boolean literal value
path.node.value = t.booleanLiteral(newValue);
}
},
// after the first transformation it changes to React.createElement
// that's why we use ObjectPropety here
ObjectProperty: path => {
const { name: currentKey } = path.node.key;
const newValue = attrs[currentKey];
if (typeof newValue !== "undefined") {
path.node.value = t.booleanLiteral(newValue);
}
},
},
};
},
],
});
return result.code || "";
};
const Playground = () => {
const [circled, setCircled] = React.useState(false);
const handleTransform = code => {
return transform(code, { circled });
};
return (
<Stack>
<LiveProvider
code={`() => <Button type="secondary" circled={false}>Primary</Button>`}
scope={{ Button, styled }}
{/* react-live already has prop which help us to do transformations */}
transformCode={handleTransform}
>
<LiveEditor />
<LiveError />
<LivePreview />
</LiveProvider>
<Switch
checked={circled}
onChange={() => {
setCircled(prev => !prev);
}}
/>
</Stack>
);
};
export default Playground;Codemods
That's one of my favorites! Codemods are really useful and it makes your developer life much easier 😁 Imagine, that you have to update a large codebase after some breaking change or just to make some large refactoring. Of course, you can use regex search and replace in your IDE, but that will work only for simple replacements, what if you have to make comprehensive refactoring? In that case, codemod is the perfect solution.
We can use a tool called jscodeshift, which runs codemodes over multiple javascript and typescript files.
I will give you an example from our codebase. We have a component called Stack and once we decided to change API, property spacing, which is responsible for spacings between child elements
and we decided to change its values to more intuitive ones.
const mediaQueries = ["mediumMobile", "largeMobile", "tablet", "desktop", "largeDesktop"];
// current values
const oldSpacings = [
"none",
"extraTight",
"tight",
"condensed",
"compact",
"natural",
"comfy",
"loose",
"extraLoose",
];
// new ones
const newSpacings = [
"none",
"XXXSmall",
"XXSmall",
"XSmall",
"small",
"medium",
"large",
"XLarge",
"XXLarge",
];
const replaceValue = value => {
if (newSpacings.findIndex(v => v === value) === -1) {
return newSpacings[oldSpacings.findIndex(v => v === value)];
}
return value;
};
function transformStackSpacing(fileInfo, api) {
const j = api.jscodeshift;
const root = j(fileInfo.source);
// basic usage on JSX element with string literal
root
.find(j.JSXOpeningElement, { name: { name: "Stack" } })
.find(j.JSXAttribute, { name: { name: "spacing" } })
.forEach(path => {
path.node.value.value = replaceValue(path.node.value.value);
});
// usage in media queries with string literal
// we also have to replaace it inside mediaQuery properties
// from <Stack spacing="compact" largeMobile={{spacing: "loose"}}> to <Stack spacing="small" largeMobile={{spacing: "XLarge"}}
mediaQueries.forEach(mq => {
const objectExpressions = root
.findJSXElements("Stack")
.find(j.JSXAttribute, { name: { name: mq }, value: { type: "JSXExpressionContainer" } })
.find(j.ObjectExpression);
// this is where Babel and TypeScript AST differ
for (const nodeType of [j.Property, j.ObjectProperty]) {
objectExpressions.find(nodeType, { key: { name: "spacing" } }).forEach(path => {
path.node.value.value = replaceValue(path.node.value.value);
});
}
});
// usage in conditional expression
root
.findJSXElements("Stack")
.find(j.JSXAttribute, { name: { name: "spacing" } })
.find(j.ConditionalExpression)
.forEach(path => {
path.node.consequent.value = replaceValue(path.node.consequent.value);
path.node.alternate.value = replaceValue(path.node.alternate.value);
});
return root.toSource();
}
module.exports = transformStackSpacing;As you can see, it's simple, it looks slightly different from the @babel/parser examples above, but once you know which node you have to find and which value to replace - it's becoming really
straightforward and easy.
Then you can just apply it to your codebase:
jscodeshift -t stack-spacing.js '../src/**/*.js' -pIt's also better to write tests before you will start to apply it to your codebase, that will also help you to check if it while working on that codemod 😉
Analytics
You can also use AST for static code analysis, without transforming it, just to find the information you need. It can be useful, especially for a design system, when you have to support multiple projects and you have to measure adoption, how many components are used, where and how. You can write the parser tool, which will walk through the code and collect information, or use an already existing solution react-scanner.
That's how we track our components, except for the number of instances, we also collect component props instances with the links to our projects where it's used. So it gives us a lot of information about the usage of our design system, helps us to make decisions about deprecations, like how the next deprecation will affect our users and etc.
And everything that with the help of AST.
PS
I hope you enjoyed that. It's a large topic and I wanted to show examples, where I found it useful, without going deep into details. If you are new to it and never used ASTs before, it's time to start! Good luck and have fun ✌️