Skip to content

Commit

Permalink
Add README
Browse files Browse the repository at this point in the history
  • Loading branch information
haroldadmin committed Apr 5, 2024
1 parent 38deb67 commit f1eb000
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 0 deletions.
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,18 @@ A parser for robots.txt files.
## Installation

_TODO_

## Usage

Use the `parse` function to convert a robots.txt string into a parse tree.

```ts
import { parse } from "robots-parser";

const res = await fetch("https://www.google.com/robots.txt");
const robotsTxt = await res.text();

const parseTree = parse(robotsTxt);

console.log(JSON.stringify(parseTree));
```
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
"name": "robots-parser",
"version": "1.0.0",
"type": "module",
"exports": "./index.ts",
"description": "Parser for robots.txt",
"scripts": {
"build": "tsc",
Expand Down
1 change: 1 addition & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export { parse } from "./parser.js";

0 comments on commit f1eb000

Please sign in to comment.