Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Management new #705

Draft
wants to merge 66 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 64 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
a80bd7b
Scaffolded React frontend with FluentUI
einari Jun 1, 2021
8d14f5b
Up and running with basics of HotChocolate GraphQL
einari Jun 1, 2021
9ba411e
Up and running with Apollo Client
einari Jun 1, 2021
7171e7a
Cleanup
einari Jun 1, 2021
e3e31db
Layout up and running
einari Jun 1, 2021
1d40636
Adding test data
einari Jun 1, 2021
7bf6150
Changing foldername to layouts
einari Jun 1, 2021
564bb11
Start of showing event handlers
einari Jun 1, 2021
d1a4d92
Testing out shimmering
einari Jun 1, 2021
7314eec
Cleanup
einari Jun 1, 2021
101349c
Merge branch 'eventhandler-service-ping-improvement' into management
einari Jun 2, 2021
159c280
Change folder + up and running with tenancy
einari Jun 2, 2021
2107d67
Basics up and running - event handlers + tenancy
einari Jun 2, 2021
56487fb
Getting ready for some metrics
einari Jun 2, 2021
1d1292d
Added ability to get a nicer view of the metrics
einari Jun 2, 2021
e35caf8
Added refresh button
einari Jun 2, 2021
10387ae
Refactor: don't cancel the source within the handler, let it be cance…
einari Jun 3, 2021
f36d575
Adding XML Doc
einari Jun 3, 2021
bab7278
Changing from Func<> to FactoryFor<> to give a better indication of w…
einari Jun 3, 2021
f94d762
Changing to FactoryFor
einari Jun 3, 2021
cd92f72
Removing semaphore
einari Jun 3, 2021
79c7a72
Using factory to create pinged connection
einari Jun 3, 2021
e69196a
Putting in sample of remote debugging
einari Jun 4, 2021
1a432a2
Merge branch '6.0.0-timeout' into management
einari Jun 4, 2021
3825504
Introducing a simple way of keeping track of event handlers for typic…
einari Jun 4, 2021
26cd88d
Exposing current state as read only
einari Jun 4, 2021
3029fda
Adding a way to get underlying abstract scoped stream processor for a…
einari Jun 4, 2021
830dcd6
Exposing functionality for reading event store tail position
einari Jun 4, 2021
7f1d182
Removing "old" uery
einari Jun 4, 2021
929f154
Adding reference to events processing
einari Jun 4, 2021
4caeaf8
Ignore enforcing of having to implement oldschool Exception constructors
einari Jun 4, 2021
c61c90e
Hooking up so tha twe get real event handlers output
einari Jun 4, 2021
bc3dce4
Displaying the correct properties + adding refresh
einari Jun 4, 2021
4794f6f
Changing default dev to port 5000
einari Jun 4, 2021
5809793
Removing - as these are now in the Web project
einari Jun 5, 2021
4e85ed3
Adding wwwroot for ignore
einari Jun 5, 2021
b7e9e2e
Adding support for default files
einari Jun 5, 2021
0aa2fe6
Putting in Management Web UI
einari Jun 5, 2021
fe05cca
Adding a simple proxy route to be used with our metrics
einari Jun 5, 2021
79acbb7
Adding middleware that makes deeplinking work into the SPA
einari Jun 6, 2021
24cb438
Update Source/Server/ProxyServerMiddleware.cs
einari Jun 6, 2021
96a007f
Xml Comments
einari Jun 6, 2021
61cafe3
Less eager Test Explorer (not draining the CPU on startup)
einari Jun 6, 2021
654c64a
Putting the version numbers into versions.props
einari Jun 6, 2021
f62d413
Show actual tail number correctly, which is count - 1
einari Jun 6, 2021
8826f81
Upgrading to latest Prometheus and leveraging their new AspNet endpoing
einari Jun 6, 2021
177c73e
Update Source/Management.GraphQL/Management.GraphQL.csproj
einari Jun 6, 2021
db42923
Update Source/Management.Web/eventHandlers/EventHandlersOverview.tsx
einari Jun 6, 2021
e74b604
Update Source/Management.WebAPI/Management.WebAPI.csproj
einari Jun 6, 2021
d112790
Taking out remote debugging config
einari Jun 7, 2021
0045ea6
Merge branch 'management' of github.com:dolittle-runtime/Runtime into…
einari Jun 7, 2021
90e9ccb
Refactoring - splitting responsibilities
einari Jun 8, 2021
67189b9
Disposing the event handler when exiting the service
einari Jun 8, 2021
ee45ed1
Moving Dispose to unregister
einari Jun 8, 2021
2f7143a
Simplifying backend for event handlers- get for all tenants, filter i…
einari Jun 8, 2021
74d5837
XML Docs
einari Jun 8, 2021
3d4d886
Cleanup
einari Jun 8, 2021
0aca585
XML Docs
einari Jun 8, 2021
18b0245
Changing route to something that makes more sense
einari Jun 8, 2021
696ec72
Fixing proxy config - its all on 5000 now
einari Jun 8, 2021
e27546a
Merge branch '6.0.0-timeout' of github.com:dolittle-runtime/Runtime i…
einari Jun 8, 2021
6593c35
Merge branch '6.0.0-timeout' into management
einari Jun 8, 2021
0d617dc
Merge branch 'master' of github.com:dolittle/Runtime into management
woksin Jun 21, 2021
4f2dfb3
Merge branch 'master' of github.com:dolittle/Runtime into management-new
woksin Jul 8, 2022
0f56a76
Delete RuntimeEnvironment.cs
woksin Jul 8, 2022
b6cdb70
Update Source/Events.Processing/EventHandlers/EventHandler.cs
woksin Jul 8, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ Benchmarks
Diagrams
.vscode
**/node_modules
**/wwwroot
.github
3 changes: 2 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,11 @@
"git.ignoreLimitWarning": true,
"dotnet-test-explorer.testProjectPath":"Specifications/**/*.csproj",
"dotnet-test-explorer.treeMode": "full",
"dotnet-test-explorer.runInParallel": true,
"dotnet-test-explorer.runInParallel": false,
"dotnet-test-explorer.showCodeLens": true,
"dotnet-test-explorer.autoExpandTree": true,
"cSpell.words": [
"fluentui",
"Grpc",
"Protobuf"
]
Expand Down
12 changes: 12 additions & 0 deletions Docker/Development/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,17 @@ WORKDIR /app/Source/Server
RUN dotnet restore
RUN dotnet publish -c "Release" -p:Version=${VERSION} -p:RuntimeIdentifier="linux-${TARGETARCH/amd64/x64}" -o out

# Web Build
FROM node:14.2.0 AS web-build
WORKDIR /app

COPY Source/Management.Web ./Source/Management.Web

WORKDIR /app/Source/Management.Web

RUN yarn
RUN yarn build

# Runtime Image
FROM mcr.microsoft.com/dotnet/aspnet:6.0
SHELL ["/bin/bash", "-c"]
Expand All @@ -25,6 +36,7 @@ ENV DOLITTLE__RUNTIME__EVENTSTORE__BACKWARDSCOMPATIBILITY__VERSION="V7"
WORKDIR /app
COPY --from=dotnet-build /app/Source/Server/out ./
COPY --from=dotnet-build /app/Source/Server/.dolittle ./.dolittle
COPY --from=web-build /app/Source/Management.Web/wwwroot ./wwwroot

# Install MongoDB dependencies
RUN apt-get update \
Expand Down
14 changes: 13 additions & 1 deletion Docker/Production/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

# DotNet Build
FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:6.0 AS dotnet-build
SHELL ["/bin/bash", "-c"]
Expand All @@ -14,6 +15,17 @@ WORKDIR /app/Source/Server
RUN dotnet restore
RUN dotnet publish -c "Release" -p:Version=${VERSION} -p:RuntimeIdentifier="linux-${TARGETARCH/amd64/x64}" -o out

# Web Build
FROM node:14.2.0 AS web-build
WORKDIR /app

COPY Source/Management.Web ./Source/Management.Web

WORKDIR /app/Source/Management.Web

RUN yarn
RUN yarn build

# Runtime Image
FROM mcr.microsoft.com/dotnet/aspnet:6.0
SHELL ["/bin/bash", "-c"]
Expand All @@ -22,7 +34,7 @@ ENV Logging__Console__FormatterName=""

WORKDIR /app
COPY --from=dotnet-build /app/Source/Server/out ./
COPY --from=dotnet-build /app/Source/Server/.dolittle ./.dolittle
COPY --from=web-build /app/Source/Management.Web/wwwroot ./wwwroot
COPY Docker/Production/.dolittle ./.dolittle

EXPOSE 9700 50052 50053 51052
Expand Down
17 changes: 17 additions & 0 deletions Source/Events.Processing/EventHandlers/EventHandler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,23 @@ public async Task<Try<IDictionary<TenantId, Try<StreamPosition>>>> ReprocessAllE
{
return ex;
}

async Task ValidateFilter()
{
_logger.ValidatingFilter(FilterDefinition.TargetStream);
var filterValidationResults = await _filterValidator.Validate(GetFilterProcessor, _cancellationTokenSource.Token).ConfigureAwait(false);

if (filterValidationResults.Any(_ => !_.Value.Succeeded))
{
var firstFailedValidation = filterValidationResults.First(_ => !_.Value.Succeeded).Value;
_logger.FilterValidationFailed(FilterDefinition.TargetStream, firstFailedValidation.FailureReason);
throw new FilterValidationFailed(FilterDefinition.TargetStream, firstFailedValidation.FailureReason);
}

var filteredStreamDefinition = new StreamDefinition(FilterDefinition);
_logger.PersistingStreamDefinition(filteredStreamDefinition.StreamId);
await _streamDefinitions.Persist(Scope, filteredStreamDefinition, _cancellationTokenSource.Token).ConfigureAwait(false);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
async Task ValidateFilter()
{
_logger.ValidatingFilter(FilterDefinition.TargetStream);
var filterValidationResults = await _filterValidator.Validate(GetFilterProcessor, _cancellationTokenSource.Token).ConfigureAwait(false);
if (filterValidationResults.Any(_ => !_.Value.Succeeded))
{
var firstFailedValidation = filterValidationResults.First(_ => !_.Value.Succeeded).Value;
_logger.FilterValidationFailed(FilterDefinition.TargetStream, firstFailedValidation.FailureReason);
throw new FilterValidationFailed(FilterDefinition.TargetStream, firstFailedValidation.FailureReason);
}
var filteredStreamDefinition = new StreamDefinition(FilterDefinition);
_logger.PersistingStreamDefinition(filteredStreamDefinition.StreamId);
await _streamDefinitions.Persist(Scope, filteredStreamDefinition, _cancellationTokenSource.Token).ConfigureAwait(false);
}

woksin marked this conversation as resolved.
Show resolved Hide resolved
}

/// <inheritdoc/>
Expand Down
29 changes: 29 additions & 0 deletions Source/Management.GraphQL/EventHandlers/EventHandler.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
// Copyright (c) Dolittle. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System;
using System.Collections.Generic;

namespace Dolittle.Runtime.Management.GraphQL.EventHandlers
{
/// <summary>
/// Represents an event handler and its current state.
/// </summary>
public class EventHandler
{
/// <summary>
/// Gets the unique identifier of the <see cref="EventHandler"/>
/// </summary>
public Guid Id { get; set; }

/// <summary>
/// Gets the scope the unique handler works on.
/// </summary>
public Guid Scope { get; set; }

/// <summary>
/// Gets the collection of <see cref="EventHandlerStatusPerTenant">statuses per tenant</see>.
/// </summary>
public IEnumerable<EventHandlerStatusForTenant> StatusPerTenant { get; set; }
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
// Copyright (c) Dolittle. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System;

namespace Dolittle.Runtime.Management.GraphQL.EventHandlers
{
/// <summary>
/// Represents the status for a specific <see cref="EventHandler"/> for a specific tenant.
/// </summary>
public class EventHandlerStatusForTenant
{
/// <summary>
/// Gets or sets the identifier of the tenant.
/// </summary>
public Guid TenantId { get; set; }

/// <summary>
/// Gets or sets the last committed event sequence number.
/// </summary>
public int LastCommittedEventSequenceNumber { get; set; }

/// <summary>
/// Gets or sets the filter position.
/// </summary>
public int FilterPosition { get; set; }

/// <summary>
/// Gets or sets the vent processor position.
/// </summary>
public int EventProcessorPosition { get; set; }
}
}
81 changes: 81 additions & 0 deletions Source/Management.GraphQL/EventHandlers/EventHandlers.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
// Copyright (c) Dolittle. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Dolittle.Runtime.ApplicationModel;
using Dolittle.Runtime.DependencyInversion;
using Dolittle.Runtime.Events.Processing.EventHandlers;
using Dolittle.Runtime.Events.Store;
using Dolittle.Runtime.Execution;
using Dolittle.Runtime.Tenancy;

namespace Dolittle.Runtime.Management.GraphQL.EventHandlers
{
/// <summary>
/// Represents an API endpoint for working with event handlers.
/// </summary>
public class EventHandlers
{
readonly IEventHandlers _eventHandlers;
readonly IExecutionContextManager _executionContextManager;
readonly IContainer _container;
readonly ITenants _tenants;

/// <summary>
/// Initializes a new instance of <see cref="EventHandlers"/>.
/// </summary>
/// <param name="eventHandlers">The runtime <see cref="IEventHandlers"/>.</param>
/// <param name="executionContextManager">The <see cref="IExecutionContextManager"/> for working with execution context.</param>
/// <param name="container">The <see cref="IContainer"/> for service location.</param>
/// <param name="tenants">The <see cref="ITenants"/> of the runtime.</param>
public EventHandlers(
IEventHandlers eventHandlers,
IExecutionContextManager executionContextManager,
IContainer container,
ITenants tenants)
{
_eventHandlers = eventHandlers;
_executionContextManager = executionContextManager;
_container = container;
_tenants = tenants;
}

/// <summary>
/// Get all event handlers and their statuses.
/// </summary>
public async Task<IEnumerable<EventHandler>> All()
{
var lastCommittedEventSequenceNumberPerTenant = new Dictionary<TenantId, EventLogSequenceNumber>();

var tasks = _tenants.All.Select(async tenant =>
{
_executionContextManager.CurrentFor(Microservice.NotSet, tenant);
var eventStore = _container.Get<FactoryFor<IEventStore>>()();
lastCommittedEventSequenceNumberPerTenant[tenant] = await eventStore.GetLastCommittedEventSequenceNumber().ConfigureAwait(false);
});
await Task.WhenAll(tasks).ConfigureAwait(false);

return _eventHandlers.All.Select(_ =>
{
var query = from filter in _.FilterStreamProcessor.StreamProcessorsPerTenant
join eventProcessor in _.EventProcessorStreamProcessor.StreamProcessorsPerTenant on filter.Key equals eventProcessor.Key
select new EventHandlerStatusForTenant
{
TenantId = filter.Key,
LastCommittedEventSequenceNumber = (int)lastCommittedEventSequenceNumberPerTenant[filter.Key].Value,
FilterPosition = (int)filter.Value.CurrentState.Position.Value,
EventProcessorPosition = (int)eventProcessor.Value.CurrentState.Position.Value
};

return new EventHandler
{
Id = _.EventProcessor,
Scope = _.Scope,
StatusPerTenant = query.ToArray()
};
});
}
}
}
29 changes: 29 additions & 0 deletions Source/Management.GraphQL/GraphQLEndpoints.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
// Copyright (c) Dolittle. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using HotChocolate.Execution.Configuration;
using Microsoft.Extensions.DependencyInjection;

namespace Dolittle.Runtime.Management.GraphQL
{
/// <summary>
/// Represents extension methods for working with exposing the GraphQL endpoints for management.
/// </summary>
public static class GraphQLEndpoints
{
/// <summary>
/// Adds the GraphQL API for management to the HotChocolate GraphQL builder.
/// </summary>
/// <param name="graphQLBuilder"><see cref="IRequestExecutorBuilder"/> to add to.</param>
/// <returns><see cref="IRequestExecutorBuilder"/> for continuation.</returns>
public static IRequestExecutorBuilder AddManagementAPI(this IRequestExecutorBuilder graphQLBuilder)
{
graphQLBuilder.AddQueryType(_ => _
.Name("Query")
.AddSubObject<EventHandlers.EventHandlers>("eventHandlers")
.AddSubObject<Tenancy.Tenants>("tenancy")
);
return graphQLBuilder;
}
}
}
18 changes: 18 additions & 0 deletions Source/Management.GraphQL/Management.GraphQL.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<Project Sdk="Microsoft.NET.Sdk">
<Import Project="../../default.props"/>

<PropertyGroup>
<AssemblyName>Dolittle.Runtime.Management.GraphQL</AssemblyName>
<RootNamespace>Dolittle.Runtime.Management.GraphQL</RootNamespace>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="HotChocolate.AspNetCore" Version="$(HotChocolateVersion)" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="../Tenancy/Tenancy.csproj"/>
<ProjectReference Include="../Events.Processing/Events.Processing.csproj"/>
</ItemGroup>

</Project>
30 changes: 30 additions & 0 deletions Source/Management.GraphQL/ObjectFieldExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
// Copyright (c) Dolittle. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System;
using HotChocolate.Types;
using Microsoft.Extensions.DependencyInjection;

namespace Dolittle.Runtime.Management.GraphQL
{
/// <summary>
/// Extension methods for <see cref="IObjectTypeDescriptor"/> and fields.
/// </summary>
public static class ObjectFieldExtensions
{
public static IServiceProvider ServiceProvider { get; set; }

/// <summary>
/// Add a sub field on a type, such as a Query / Mutation root object with a given name that will resolve to the given type.
/// </summary>
/// <param name="parent">Parent type (Query/Mutation).</param>
/// <param name="name">Name of the field.</param>
/// <typeparam name="T">Type that the field should resolve to.</typeparam>
/// <returns><see cref="IObjectTypeDescriptor"/> for continuation.</returns>
public static IObjectTypeDescriptor AddSubObject<T>(this IObjectTypeDescriptor parent, string name)
{
parent.Field(name).Type(typeof(EventHandlers.EventHandlers)).Resolve(_ => ServiceProvider.GetService<T>());
return parent;
}
}
}
32 changes: 32 additions & 0 deletions Source/Management.GraphQL/Tenancy/Tenants.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// Copyright (c) Dolittle. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System;
using System.Collections.Generic;
using System.Linq;
using Dolittle.Runtime.Tenancy;

namespace Dolittle.Runtime.Management.GraphQL.Tenancy
{
/// <summary>
/// Represents an API for getting all tenants.
/// </summary>
public class Tenants
{
readonly ITenants _tenants;

/// <summary>
/// Initializes a new instance of <see cref="Tenants"/>.
/// </summary>
/// <param name="tenants">The runtime <see cref="ITenants"/>.</param>
public Tenants(ITenants tenants)
{
_tenants = tenants;
}

/// <summary>
/// Gets all the tenants registered with the runtime.
/// </summary>
public IEnumerable<Guid> All() => _tenants.All.Select(_ => _.Value);
}
}
Loading