Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] add support for TypedArrays/arrayBuffer/nodejs' Buffer #716

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

dotnetCarpenter
Copy link

Currently failing to implement tests for TypedArray.

In assert.strictEqual (equals (actual) (expected), true); the
Setoid.test test fails unconditionaly for a TypedArray (Uint8Array).

Need help ;)

Reproduce:

const S = require ('./internal/sanctuary');
const eq = require ('./internal/eq');

eq (S.unchecked.drop (1) (new Uint8Array([10, 20, 30, 40, 50]))) (S.unchecked.Just (new Uint8Array([20, 30, 40, 50])));

PS. For now my goal is to support various Buffers for unchecked functions, since it seems like this should be opt-in and I don't know how to implement support in https://github.com/sanctuary-js/sanctuary-def yet.

Currently failing to implement tests for TypedArray.

In `assert.strictEqual (equals (actual) (expected), true);` the
`Setoid.test` test fails unconditionaly for a TypedArray (`Uint8Array`).

Need help ;)
@dotnetCarpenter dotnetCarpenter changed the title [WIP] add support for TypedArrays/arraBuffer/nodejs' Buffer [WIP] add support for TypedArrays/arrayBuffer/nodejs' Buffer Aug 16, 2021
@davidchambers
Copy link
Member

I don't think this makes sense.

Here is the type of take:

take :: (Applicative f, Foldable f, Monoid (f a)) => Integer -> f a -> Maybe (f a)

We could specialize this by replacing f with Array and a with String:

take :: Integer -> Array String -> Maybe (Array String)

Doing the same with Uint8Array results in a nonsensical type:

take :: Integer -> Uint8Array String -> Maybe (Uint8Array String)

Uint8Array is a type rather than a type constructor: these arrays only ever contain 8-bit unsigned integers.

One could define a specialized function:

takeUint8 :: Integer -> Uint8Array -> Maybe Uint8Array

@dotnetCarpenter
Copy link
Author

dotnetCarpenter commented Aug 16, 2021

@davidchambers How would I go about implementing specialised functions for Sanctuary?

Would TypedArray encompass all of Int8Array, Uint8Array, Uint8ClampedArray, Int16Array, Uint16Array, Int32Array, Uint32Array, Float32Array, Float64Array, BigInt64Array and BigUint64Array?

takeTypedArray :: Integer -> TypedArray -> Maybe TypedArray

@davidchambers
Copy link
Member

The hypothetical TypedArray type is problematic because it would allow takeTypedArray to take a Uint8Array but return a Maybe Float64Array.

The simplest solution would be to define a specialized function for each type.

@dotnetCarpenter
Copy link
Author

So that would be 11 specialized functions, just for take?

takeInt8Array :: Integer -> Int8Array -> Maybe Int8Array
takeUint8Array :: NonNegativeInteger -> Uint8Array -> Maybe Uint8Array
takeUint8ClampedArray :: NonNegativeInteger -> Uint8ClampedArray -> Maybe Uint8ClampedArray
takeInt16Array :: Integer -> Int16Array -> Maybe Int16Array
takeUint16Array :: NonNegativeInteger -> Uint16Array -> Maybe Uint16Array
takeInt32Array :: Integer -> Int32Array -> Maybe Int32Array
takeUint32Array :: NonNegativeInteger -> Uint32Array -> Maybe Uint32Array
takeFloat32Array :: ValidNumber -> Float32Array -> Maybe Float32Array
takeFloat64Array :: ValidNumber -> Float64Array -> Maybe Float64Array

Does Sanctuary support BigInt?

takeBigInt64Array :: BigInt -> BigInt64Array -> Maybe BigInt64Array
takeBigUint64Array :: NonNegativeBigInt -> BigUint64Array -> Maybe BigUint64Array

But actually, before going down this rabbit hole. I'm much more interested in support for Nodejs' Buffer since I deal with a database that returns Buffers.

takeBuffer :: NonNegativeInteger -> Buffer -> Maybe Buffer

dropBuffer :: NonNegativeInteger -> Buffer -> Maybe Buffer

takeLastBuffer :: NonNegativeInteger -> Buffer -> Maybe Buffer

dropLastBuffer :: NonNegativeInteger -> Buffer -> Maybe Buffer

takeWhileBuffer :: (a -> Boolean) -> Buffer -> Buffer

dropWhileBuffer :: (a -> Boolean) -> Buffer -> Buffer

Ref: NonNegativeInteger

@dotnetCarpenter
Copy link
Author

I tried to look through some code a few months back to find context for this. I did not find anything.

But as I recall, I got a Buffer from the database and wanted to use S.unchecked.drop but the Array,isArray check in _takeDrop would not let me, so I used .slice instead without a Maybe.

Changing that one line, made it all work in Sanctuary but .... here I am.

@Avaq
Copy link
Member

Avaq commented Aug 16, 2021

If I'd want to take from a Buffer, I'd probably do something like:

// firstFiveBytes :: Buffer -> Maybe Buffer
const firstFiveBytes = S.pipe ([
  Array.from,
  S.take (5),
  S.map (Buffer.from),
])

@dotnetCarpenter
Copy link
Author

@Avaq I'm imagining that working with Buffers without converting them to Array would be faster overall. But I should write some code to compare with.

There is only one line that need to change in Sanctuary in order to support Buffers, so it should be fairly easy to compare.

But it won't be today :)

@imcotton
Copy link

imcotton commented Aug 26, 2021

Quick note: slice creates a copy of internal memory v.s. subarray keeps its ArrayBuffer untouched (shared).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants