Skip to content

Commit

Permalink
Merge branch 'main' into lane/checks-workflow
Browse files Browse the repository at this point in the history
  • Loading branch information
lanesawyer committed Jun 18, 2024
2 parents 52d41af + b244c46 commit af52e33
Show file tree
Hide file tree
Showing 107 changed files with 12,126 additions and 9,249 deletions.
8 changes: 8 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"printWidth": 120,
"trailingComma": "es5",
"tabWidth": 4,
"singleQuote": true,
"semi": true,
"singleAttributePerLine": true
}
50 changes: 25 additions & 25 deletions LICENSE.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,33 @@
Allen Institute Software License – This software license is the 2-clause BSD
license plus clause a third clause that prohibits redistribution and use for
commercial purposes without further permission.
Allen Institute Software License – This software license is the 2-clause BSD
license plus clause a third clause that prohibits redistribution and use for
commercial purposes without further permission.

Copyright © 2019. Allen Institute. All rights reserved.
Copyright © 2019. Allen Institute. All rights reserved.

Redistribution and use in source and binary forms, with or without
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

3. Redistributions and use for commercial purposes are not permitted without
the Allen Institute’s written permission. For purposes of this license,
commercial purposes are the incorporation of the Allen Institute's software
into anything for which you will charge fees or other compensation or use of
the software to perform a commercial service for a third party. Contact
[email protected] for commercial licensing opportunities.
3. Redistributions and use for commercial purposes are not permitted without
the Allen Institute’s written permission. For purposes of this license,
commercial purposes are the incorporation of the Allen Institute's software
into anything for which you will charge fees or other compensation or use of
the software to perform a commercial service for a third party. Contact
[email protected] for commercial licensing opportunities.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,22 @@
# Allen Institute Vis

This repository contains a collection of packages for building scalable visualization tools at the Allen Institute. It houses the `vis-[package name]` packages on the Allen Institute GitHub organization's NPM package registry.

# Available Packages
- `vis-geometry`: A collection of vector functions for 2D and 3D geometry
- `scatterbrain`: A collection of useful utitilites used to build our big-data, scalable, scatterplot tool "Scatterbrain" in the Brain Knowledge Platform

- `vis-geometry`: A collection of vector functions for 2D and 3D geometry
- `scatterbrain`: A collection of useful utitilites used to build our big-data, scalable, scatterplot tool "Scatterbrain" in the Brain Knowledge Platform

# Examples

See the `apps` directory for example projects using the packages. Over time, these examples will become simpler as the base tooling becomes more mature.

For details on running or adding new examples, see the `docs/examples.md` file.

# Contributing

Contributions are welcome! We're currently breaking apart the Scattebrain component into smaller, more manageable packages. If you have a package that you think would be useful to others, please open a PR.

# Using the Libraries

See the `docs/using-packages.md` file for information on how to use the packages in your own projects.
55 changes: 27 additions & 28 deletions apps/common/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,35 +3,34 @@
"version": "0.0.1",
"HEY": "NOTE! DO not use this package directly - its just for sharing code between our little apps",
"contributors": [
{
"name": "Lane Sawyer",
"email": "[email protected]"
},
{
"name": "James Gerstenberger",
"email": "[email protected]"
},
{
"name": "Noah Shepard",
"email": "[email protected]"
},
{
"name": "Skyler Moosman",
"email": "[email protected]"
},
{
"name": "Su Li",
"email": "[email protected]"
}
{
"name": "Lane Sawyer",
"email": "[email protected]"
},
{
"name": "James Gerstenberger",
"email": "[email protected]"
},
{
"name": "Noah Shepard",
"email": "[email protected]"
},
{
"name": "Skyler Moosman",
"email": "[email protected]"
},
{
"name": "Su Li",
"email": "[email protected]"
}
],
"private": true,
"dependencies": {
"@alleninstitute/vis-geometry": "workspace:*",
"@alleninstitute/vis-scatterbrain": "workspace:*",
"@types/lodash": "^4.14.202",
"lodash": "^4.17.21",
"regl": "^2.1.0",
"zarr": "^0.6.2"

"@alleninstitute/vis-geometry": "workspace:*",
"@alleninstitute/vis-scatterbrain": "workspace:*",
"@types/lodash": "^4.14.202",
"lodash": "^4.17.21",
"regl": "^2.1.0",
"zarr": "^0.6.2"
}
}
}
26 changes: 11 additions & 15 deletions apps/common/src/loaders/ome-zarr/fetchSlice.worker.ts
Original file line number Diff line number Diff line change
@@ -1,30 +1,26 @@

// a web-worker which fetches slices of data, decodes them, and returns the result as a flat float32 array, using transferables
import type { NestedArray, TypedArray } from 'zarr'
import { getSlice, type ZarrDataset, type ZarrRequest } from "./zarr-data";
import type { NestedArray, TypedArray } from 'zarr';
import { getSlice, type ZarrDataset, type ZarrRequest } from './zarr-data';

const ctx = self;
type ZarrSliceRequest = {
id: string;
type: 'ZarrSliceRequest'
metadata: ZarrDataset
req: ZarrRequest,
layerIndex: number
}
type: 'ZarrSliceRequest';
metadata: ZarrDataset;
req: ZarrRequest;
layerIndex: number;
};
function isSliceRequest(payload: any): payload is ZarrSliceRequest {
return typeof payload === 'object' && payload['type'] === 'ZarrSliceRequest';
}
ctx.onmessage = (msg: MessageEvent<unknown>) => {
const { data } = msg;
if (isSliceRequest(data)) {
const { metadata, req, layerIndex, id } = data;
getSlice(metadata, req, layerIndex).then((result: {
shape: number[],
buffer: NestedArray<TypedArray>
}) => {
getSlice(metadata, req, layerIndex).then((result: { shape: number[]; buffer: NestedArray<TypedArray> }) => {
const { shape, buffer } = result;
const R = new Float32Array(buffer.flatten());
ctx.postMessage({ type: 'slice', id, shape, data: R }, { transfer: [R.buffer] })
})
ctx.postMessage({ type: 'slice', id, shape, data: R }, { transfer: [R.buffer] });
});
}
}
};
30 changes: 18 additions & 12 deletions apps/common/src/loaders/ome-zarr/sliceWorkerPool.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { uniqueId } from "lodash";
import type { ZarrDataset, ZarrRequest } from "./zarr-data";
import { uniqueId } from 'lodash';
import type { ZarrDataset, ZarrRequest } from './zarr-data';

type PromisifiedMessage = {
requestCacheKey: string;
Expand All @@ -8,15 +8,15 @@ type PromisifiedMessage = {
promise?: Promise<Slice> | undefined;
};
type ExpectedResultSlice = {
type: 'slice',
type: 'slice';
id: string;
} & Slice;
type Slice = {
data: Float32Array;
shape: number[]
}
shape: number[];
};
function isExpectedResult(obj: any): obj is ExpectedResultSlice {
return (typeof obj === 'object' && 'type' in obj && obj.type === 'slice')
return typeof obj === 'object' && 'type' in obj && obj.type === 'slice';
}
export class SliceWorkerPool {
private workers: Worker[];
Expand All @@ -26,7 +26,7 @@ export class SliceWorkerPool {
this.workers = new Array(size);
for (let i = 0; i < size; i++) {
this.workers[i] = new Worker(new URL('./fetchSlice.worker.ts', import.meta.url), { type: 'module' });
this.workers[i].onmessage = (msg) => this.handleResponse(msg)
this.workers[i].onmessage = (msg) => this.handleResponse(msg);
}
this.promises = {};
this.which = 0;
Expand All @@ -37,14 +37,14 @@ export class SliceWorkerPool {
if (isExpectedResult(payload)) {
const prom = this.promises[payload.id];
if (prom) {
const { data, shape } = payload
const { data, shape } = payload;
prom.resolve({ data, shape });
delete this.promises[payload.id]
delete this.promises[payload.id];
}
}
}
private roundRobin() {
this.which = (this.which + 1) % this.workers.length
this.which = (this.which + 1) % this.workers.length;
}
requestSlice(dataset: ZarrDataset, req: ZarrRequest, layerIndex: number) {
const reqId = uniqueId('rq');
Expand All @@ -57,7 +57,13 @@ export class SliceWorkerPool {
reject,
promise: undefined, // ill get added to the map once I am fully defined!
};
this.workers[this.which].postMessage({ id: reqId, type: 'ZarrSliceRequest', metadata: dataset, req, layerIndex });
this.workers[this.which].postMessage({
id: reqId,
type: 'ZarrSliceRequest',
metadata: dataset,
req,
layerIndex,
});
this.roundRobin();
});
this.promises[reqId].promise = eventually;
Expand All @@ -72,4 +78,4 @@ export function getSlicePool() {
slicePool = new SliceWorkerPool(16);
}
return slicePool;
}
}
Loading

0 comments on commit af52e33

Please sign in to comment.