Skip to content

Commit

Permalink
Feature/bmspt 275 bcfzip with stream (#17)
Browse files Browse the repository at this point in the history 
* [BMSPT-275] implementation of bcf zip creation with memory stream

* [BMSPT-275] refactor & reorganization

* [BMSPT-275] added unit tests

* Simplified IConverter API. Added support for predefined output streams.

* [BMSPT-297] Checking incoming streams if they can be used for the given use case. Added documentation.

---------

Co-authored-by: DanielLepold <[email protected]>
Co-authored-by: Adam Eri <[email protected]>
  • Loading branch information
@BalintBende @DanielLepold @eriadam
3 people authored Jun 13, 2024
1 parent d911a85 commit 9b4da4d
Show file tree
Hide file tree
Showing 17 changed files with 597 additions and 222 deletions.
91 changes: 38 additions & 53 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,46 +1,19 @@
# bcf-toolkit

A .NET library and a command line utility for converting `BCF` (Building
Collaboration Format) files into `json` and vice versa.

The tool converts `BCF` information across formats and versions.

## Requirements

- dotnet 8

### CLI

The command line interface accepts 2 arguments:
* the source bcf file or json folder [REQUIRED]
* the target bcf file or json folder [REQUIRED]
~~* the target version of bcf [OPTIONAL]~~

The json representation is one file for every `Markup`, while the BCF format
is a zipped file as per the standard.
This C# NuGet library allows you to easily build up and convert data into BCF
files. It gives you a straightforward API to build your BCF objects exactly how
you want in your order.

```
~ bcf-toolkit /path/to/source.bcfzip /path/to/target/json/folder
~ bcf-toolkit /path/to/source/json/folder /path/to/target.bcfzip
```

### As A Library
This C# NuGet library allows the user to easily build up and convert data into
BCF files. It gives a straightforward API to build the BCF objects exactly in
the order of the user's choice.

#### Installation
`Smino.Bcf.Toolkit` library can be installed via NuGet Package Manager or by adding
it to the project's .csproj file.
## Installation
You can install the `BcfToolkit` library via NuGet Package Manager or by adding
it to your project's .csproj file.
```
nuget install Smino.Bcf.Toolkit
```

#### Usage
##### Creating BCF objects
To create a BCF Model, `BcfBuilder` class can be used. Then, various
functions provided by the builder can be used to fulfill the BCF model objects.
## Usage

### Creating BCF objects
To create a BCF Model, `BcfBuilder` class can be used. Then, various
functions provided by the builder can be used to fulfill the BCF model objects.

Here are some examples:

Expand Down Expand Up @@ -99,9 +72,9 @@ var bcf = await builder
.BuildFromStream(stream);
```

The default builders can be used if the user prefers not to deal with filling
the required fields. The `builder.WithDefaults()` function serves this.
However in certain cases the user may need to replace the component IDs of IFC
The default builders can be used if the user prefers not to deal with filling
the required fields. The `builder.WithDefaults()` function serves this.
However in certain cases the user may need to replace the component IDs of IFC
objects with the actual GUIDs during the build process.

```csharp
Expand All @@ -112,9 +85,10 @@ var bcf = builder
.WithDefaults()
.Build();
```
##### Using BCF worker
The worker is implemented to use predefined workflows to convert `BCF` files
into `json` and back. The function decides which workflow must be used according

### Using BCF worker
The worker is implemented to use predefined workflows to convert `BCF` files
into `json` and back. The function decides which workflow must be used according
to the source and target. If the source ends with `.bcfzip` the converter uses
the `BcfZipToJson` for example.

Expand All @@ -124,7 +98,7 @@ using BcfToolkit;
var worker = new Worker();
await worker.Convert(source, target);
```
The exact converter can be called directly as well for both converting
The exact converter can be called directly as well for both converting
directions, `BCF` into `json` and back.

```csharp
Expand All @@ -142,7 +116,7 @@ converter.JsonToBcfZip(source, target);
```

Furthermore `BCF` archive can be consumed as a stream. The version of the source
is established by the code, then the class lets the nested converter object to
is established by the code, then the class lets the nested converter object to
do the conversion to BCF 3.0 accordingly.

```csharp
Expand All @@ -151,7 +125,7 @@ using BcfToolkit;
await using var stream = new FileStream(source, FileMode.Open, FileAccess.Read);

var worker = new Worker();
var bcf = await worker.BuildBcfFromStream(stream);
var bcf = await worker.BcfFromStream(stream);
```

The worker can return a file stream from the specified instance of the bcf
Expand All @@ -162,14 +136,28 @@ stream after the usage is the responsibility of the caller.
using BcfToolkit;

var worker = new Worker();
var stream = await worker.ToBcfStream(bcf, BcfVersionEnum.Bcf30);
var stream = await worker.ToBcf(bcf, BcfVersionEnum.Bcf30);
// custom code to use the stream...
await stream.FlushAsync();
```

It is also possible to define the output stream to which the results will
be written.

```csharp
using BcfToolkit;

var worker = new Worker();
var outputStream = new MemoryStream();
worker.ToBcf(bcf, BcfVersionEnum.Bcf30, outputStream);
// custom code to use the stream...
await outputStream.FlushAsync();
```


## File Structure

The structure of the BCF is per [the standard][3]. There is, however, no
The structure of the BCF is per [the standard][3]. There is, however, no
standard for the JSON format other than the [BCF API specification][4].

The naming convention for this converter is taken from the BCF API, but the
Expand All @@ -194,7 +182,7 @@ named using the `uuid` of the `Topic` within.

## Development

The development of the tool is ongoing, the table below shows the currently
The development of the tool is ongoing, the table below shows the currently
completed features.

| | BCF 2.0 | BCF 2.1 | BCF 3.0 | JSON 2.0 | JSON 2.1 | JSON 3.0 |
Expand All @@ -216,9 +204,6 @@ The models for the BCF in-memory representation were auto-created from the

To publish, run the script at `dist/publish.sh`.

### TODO

- profile memory and CPU usage

### Contribution

Expand Down
21 changes: 20 additions & 1 deletion bcf-toolkit.sln.DotSettings.user
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,26 @@




<s:String x:Key="/Default/Environment/UnitTesting/UnitTestSessionStore/Sessions/=56d6a9c1_002Dffbd_002D4874_002Daebf_002Da47bc484bfc5/@EntryIndexedValue">&lt;SessionState ContinuousTestingMode="0" IsActive="True" Name="All tests from Solution #3" xmlns="urn:schemas-jetbrains-com:jetbrains-ut-session"&gt;
&lt;Solution /&gt;
&lt;/SessionState&gt;</s:String>
<s:String x:Key="/Default/Environment/UnitTesting/UnitTestSessionStore/Sessions/=65ccaa39_002D16f7_002D40f7_002Daabb_002Daa7b122623e4/@EntryIndexedValue">&lt;SessionState ContinuousTestingMode="0" Name="All tests from Solution #2" xmlns="urn:schemas-jetbrains-com:jetbrains-ut-session"&gt;
&lt;Solution /&gt;
&lt;/SessionState&gt;</s:String>



<s:String x:Key="/Default/Environment/UnitTesting/UnitTestSessionStore/Sessions/=73a22876_002D8258_002D4782_002Daf2e_002Dcdc29fed2eab/@EntryIndexedValue">&lt;SessionState ContinuousTestingMode="0" Name="All tests from Solution #2" xmlns="urn:schemas-jetbrains-com:jetbrains-ut-session"&gt;
&lt;Solution /&gt;
&lt;/SessionState&gt;</s:String>
<s:String x:Key="/Default/Environment/UnitTesting/UnitTestSessionStore/Sessions/=7889e21b_002Dbd9b_002D4fd2_002Db67f_002D8c109b99a118/@EntryIndexedValue">&lt;SessionState ContinuousTestingMode="0" IsActive="True" Name="All tests from Solution #3" xmlns="urn:schemas-jetbrains-com:jetbrains-ut-session"&gt;
&lt;Solution /&gt;
&lt;/SessionState&gt;</s:String>

<s:String x:Key="/Default/Environment/UnitTesting/UnitTestSessionStore/Sessions/=bbb7e50b_002Dc35c_002D402f_002Dbc25_002D4c70e2798aa6/@EntryIndexedValue">&lt;SessionState ContinuousTestingMode="0" IsActive="True" Name="All tests from Solution" xmlns="urn:schemas-jetbrains-com:jetbrains-ut-session"&gt;
&lt;Solution /&gt;
&lt;/SessionState&gt;</s:String>



Expand All @@ -33,7 +49,10 @@




<s:String x:Key="/Default/Environment/UnitTesting/UnitTestSessionStore/Sessions/=dce9242c_002D72ee_002D4307_002D813f_002De76e96a58816/@EntryIndexedValue">&lt;SessionState ContinuousTestingMode="0" IsActive="True" Name="All tests from Solution" xmlns="urn:schemas-jetbrains-com:jetbrains-ut-session"&gt;
&lt;Solution /&gt;
&lt;/SessionState&gt;</s:String>




Expand Down
77 changes: 47 additions & 30 deletions src/bcf-toolkit/Converter/Bcf21/Converter.cs
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Threading.Tasks;
using BcfToolkit.Builder.Bcf21;
using BcfToolkit.Utils;
Expand All @@ -15,14 +16,13 @@ namespace BcfToolkit.Converter.Bcf21;
/// JSON, and BCFzip.
/// </summary>
public class Converter : IConverter {

private BcfBuilder _builder = new();

/// <summary>
/// Defines the converter function, which must be used for converting the
/// BCF object to the targeted version.
/// </summary>
private readonly Dictionary<BcfVersionEnum, Func<Bcf, IBcf>> _converterFnMapper =
private readonly Dictionary<BcfVersionEnum, Func<Bcf, IBcf>> _converterFn =
new() {
[BcfVersionEnum.Bcf21] = b => b,
[BcfVersionEnum.Bcf30] = SchemaConverterToBcf30.Convert
Expand All @@ -32,29 +32,42 @@ public class Converter : IConverter {
/// Defines the file writer function which must be used for write the BCF
/// object to the targeted version.
/// </summary>
private readonly Dictionary<BcfVersionEnum, Func<IBcf, string, bool, Task<string>>> _writerFnMapper =
new() {
[BcfVersionEnum.Bcf21] = FileWriter.WriteBcf,
[BcfVersionEnum.Bcf30] = Bcf30.FileWriter.WriteBcf
};
private readonly Dictionary<BcfVersionEnum, Func<IBcf, Task<Stream>>>
_writerFn =
new() {
[BcfVersionEnum.Bcf21] = FileWriter.SerializeAndWriteBcf,
[BcfVersionEnum.Bcf30] = Bcf30.FileWriter.SerializeAndWriteBcf
};

public async Task BcfZipToJson(Stream source, string targetPath) {
/// <summary>
/// Defines the stream writer function which must be used for write the BCF
/// object to the targeted version.
/// </summary>
private readonly Dictionary<BcfVersionEnum, Action<IBcf, ZipArchive>>
_streamWriterFn =
new() {
[BcfVersionEnum.Bcf21] = FileWriter.SerializeAndWriteBcfToStream,
[BcfVersionEnum.Bcf30] = Bcf30.FileWriter.SerializeAndWriteBcfToStream
};

public async Task BcfToJson(Stream source, string targetPath) {
var bcf = await _builder.BuildFromStream(source);
await FileWriter.WriteJson(bcf, targetPath);
await FileWriter.WriteBcfToJson(bcf, targetPath);
}

public async Task BcfZipToJson(string sourcePath, string targetPath) {
public async Task BcfToJson(string sourcePath, string targetPath) {
try {
await using var fileStream =
new FileStream(sourcePath, FileMode.Open, FileAccess.Read);
await BcfZipToJson(fileStream, targetPath);
await BcfToJson(fileStream, targetPath);
}
catch (Exception ex) {
throw new ArgumentException($"Source path is not readable. {ex.Message}", ex);
throw new ArgumentException($"Source path is not readable. {ex.Message}",
ex);
}
}

public async Task JsonToBcfZip(string source, string target) {
public async Task JsonToBcf(string source, string target) {
// Project is optional
var projectPath = $"{source}/project.json";
var project = Path.Exists(projectPath)
Expand All @@ -69,39 +82,43 @@ public async Task JsonToBcfZip(string source, string target) {
Version = new Version()
};

await FileWriter.WriteBcf(bcf, target);
await FileWriter.SerializeAndWriteBcfToFolder(bcf, target);
}

public async Task<Stream> ToBcfStream(IBcf bcf, BcfVersionEnum targetVersion) {
var converterFn = _converterFnMapper[targetVersion];
public async Task<Stream> ToBcf(IBcf bcf, BcfVersionEnum targetVersion) {
var converterFn = _converterFn[targetVersion];
var convertedBcf = converterFn((Bcf)bcf);

var workingDir = Directory.GetCurrentDirectory();
var tmpBcfTargetPath = workingDir + $"/{Guid.NewGuid()}.bcfzip";
var writerFn = _writerFnMapper[targetVersion];
var writerFn = _writerFn[targetVersion];
return await writerFn(convertedBcf);
}

// keep the tmp files till the stream is created
var tmpFolder = await writerFn(convertedBcf, tmpBcfTargetPath, false);
var stream = new FileStream(tmpBcfTargetPath, FileMode.Open, FileAccess.Read);
public void ToBcf(IBcf bcf, BcfVersionEnum targetVersion, Stream stream) {

Directory.Delete(tmpFolder, true);
File.Delete(tmpBcfTargetPath);
if (!stream.CanWrite) {
throw new ArgumentException("Stream is not writable.");
}

var converterFn = _converterFn[targetVersion];
var convertedBcf = converterFn((Bcf)bcf);

return stream;
var writerFn = _streamWriterFn[targetVersion];
var zip = new ZipArchive(stream, ZipArchiveMode.Create, true);
writerFn(convertedBcf, zip);
}

public Task ToBcfZip(IBcf bcf, string target) {
return FileWriter.WriteBcf((Bcf)bcf, target);
public Task ToBcf(IBcf bcf, string target) {
return FileWriter.SerializeAndWriteBcfToFolder((Bcf)bcf, target);
}

public Task ToJson(IBcf bcf, string target) {
return FileWriter.WriteJson((Bcf)bcf, target);
return FileWriter.WriteBcfToJson((Bcf)bcf, target);
}

public async Task<T> BuildBcfFromStream<T>(Stream stream) {
public async Task<T> BcfFromStream<T>(Stream stream) {
var bcf = await _builder.BuildFromStream(stream);
var targetVersion = BcfVersion.TryParse(typeof(T));
var converterFn = _converterFnMapper[targetVersion];
var converterFn = _converterFn[targetVersion];
return (T)converterFn(bcf);
}
}
Loading

0 comments on commit 9b4da4d

Please sign in to comment.