New way to readLines?

[alexgleason]

readLines is deprecated, and says:

(will be removed after 1.0.0) Use the {@link https://developer.mozilla.org/en-US/docs/Web/API/Streams_API | Web Streams API} instead.

But how do you use the Web Streams API with Deno.stdin and Deno.open?

Readlines is very convenient, and I haven't figured out how to update my code yet, eg:

async function* readStdin(): AsyncGenerator<InputMessage> {
  for await (const line of readLines(Deno.stdin)) {
    try {
      yield JSON.parse(line);
    } catch (e) {
      console.error(line);
      throw e;
    }
  }
}

[bartlomieju]

CC @iuioiua

[magurotuna]

Here's what I'd do when I want to parse jsonl file from the filesystem:

import { TextLineStream } from "jsr:@std/[email protected]/text-line-stream";
import { JsonParseStream } from "jsr:@std/[email protected]/json-parse-stream";

using f = await Deno.open("./example.jsonl");
const readable = f.readable
  .pipeThrough(new TextDecoderStream()) // decode Uint8Array to string
  .pipeThrough(new TextLineStream()) // split string line by line
  .pipeThrough(new JsonParseStream()); // parse each chunk as JSON

for await (const data of readable) {
  console.log(data.wins);
}

Result:

❯ cat example.jsonl
───────┬────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: example.jsonl
───────┼────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {"name": "Gilbert", "wins": [["straight", "7♣"], ["one pair", "10♥"]]}
   2   │ {"name": "Alexa", "wins": [["two pair", "4♠"], ["two pair", "9♠"]]}
   3   │ {"name": "May", "wins": []}
   4   │ {"name": "Deloise", "wins": [["three of a kind", "5♣"]]}
───────┴────────────────────────────────────────────────────────────────────────────────────────────────

❯ deno run --allow-read=. handle_jsonl.ts
[ [ "straight", "7♣" ], [ "one pair", "10♥" ] ]
[ [ "two pair", "4♠" ], [ "two pair", "9♠" ] ]
[]
[ [ "three of a kind", "5♣" ] ]

So basically idea here is that you can use TextDecoderStream and TextLineStream to achieve what readLines does.

[alexgleason]

Thank you @magurotuna! That worked for me. And now I understand how powerful the Stream API is.

Although, it's quote verbose for the common use-case of just reading lines. I would wrap it in a function. Maybe I would name the function readLines. And then I would consider adding it to the @std/io library. 😂

[iuioiua]

Sorry for not getting back, @alexgleason, and thank you for answering, @magurotuna. Yep, the answer provided is correct. What's nice about the current std/streams package is that it allows for these modular, small utilities that make this code possible.

The key takeaway here is that we should improve our deprecation documentation. I've created denoland/std#4636 to track doing so.

[johndeighan]

You should specify where to get TextDecoderStream from. I found this works:

import {TextDecoderStream} from "node:stream/web"

[alexgleason]

@johndeighan It's a global. You don't need to import it.