Releases: openziti/ziti
v1.2.2
Release 1.2.2
What's New
- Bug fixes and continuing progress on controller HA
Component Updates and Bug Fixes
-
github.com/openziti/secretstream: v0.1.25 -> v0.1.26
-
github.com/openziti/storage: v0.3.6 -> v0.3.8
- Issue #87 - negative URL filter returns incorrect results
-
github.com/openziti/ziti: v1.2.1 -> v1.2.2
- Issue #2559 - expired JWTs are allowed to enroll
- Issue #2543 - Support adding adding an uninitialized node to a cluster (rather than the reverse)
v1.2.1
Release 1.2.1
What's New
- Bug fixes and continuing progress on controller HA
Component Updates and Bug Fixes
- github.com/openziti/agent: v1.0.19 -> v1.0.20
- github.com/openziti/channel/v3: v3.0.10 -> v3.0.16
- github.com/openziti/foundation/v2: v2.0.50 -> v2.0.52
- github.com/openziti/identity: v1.0.88 -> v1.0.90
- github.com/openziti/metrics: v1.2.59 -> v1.2.61
- github.com/openziti/runzmd: v1.0.53 -> v1.0.55
- github.com/openziti/storage: v0.3.2 -> v0.3.6
- github.com/openziti/transport/v2: v2.0.150 -> v2.0.153
- github.com/openziti/ziti: v1.2.0 -> v1.2.1
- Issue #2543 - Support adding adding an uninitialized node to a cluster (rather than the reverse)
- Issue #2541 - Add cluster id, to prevent merging disparate clusters
- Issue #2532 - When adding an existing HA cluster member, remove/add if suffrage has changed
- Issue #2217 - Controller list is empty until peers connect
- Issue #2533 - Handle concurrent raft connections
- Issue #2534 - Ziti ID with leading hyphen causes command-line parameter ambiguity
- Issue #2528 - Updated router costs are not use when evaluating current path cost in the context of smart rerouting
v1.2.0
Release 1.2.0
What's New
- New Router Metrics
- Changes to identity connect status
- HA Bootstrap Changes
- Connect Events
- SDK Events
- Bug fixes and other HA work
New Router Metrics
The following new metrics are available for edge routers:
- edge.connect.failures - meter tracking failed connect attempts from sdks
This tracks failures to not having a valid token. Other failures which
happen earlier in the connection process may not be tracked here. - edge.connect.successes - meter tracking successful connect attempts from sdks
- edge.disconnects - meter tracking disconnects of previously successfully connected sdks
- edge.connections - gauge tracking count of currently connected sdks
Identity Connect Status
Ziti tracks whether an identity is currently connected to an edge router.
This is the hasEdgeRouterConnection
field on Identity.
Identity connection status used to be driven off of heartbeats from the edge router.
This feature doesn't work correctly when running with controller HA.
To address this, while also providing more operation insight, connect events were added
(see below for more details on the events themselves).
The controller can be configured to use status from heartbeats, connect events or both.
If both are used as source, then if either reports the identity as connected, then it
will show as connected. This is intended for when you have a mix of routers and they
don't all yet supported connect events.
The controller now also aims to be more precise about identity state. There is a new
field on Identity: edgeRouterConnectionStatus
. This field can have one of three
values:
- offline
- online
- unknown
If the identity is reported as connected to any ER, it will be marked as online
.
If the identity has been reported as connected, but the reporting ER is now
offline, the identity may still be connected to the ER. While in this state
it will be marked as 'unknown'. After a configurable interval, it will be marked
as offline.
New controller config options:
identityStatusConfig:
# valid values ['heartbeats', 'connect-events', 'hybrid']
# defaults to 'hybrid' for now
source: connect-events
# determines how often we scan for disconnected routers
# defaults to 1 minute
scanInterval: 1m
# determines how long an identity will stay in unknown status before it's marked as offline
# defaults to 5m
unknownTimeout: 5m
HA Bootstrapping Changes
Previously bootstrapping the RAFT cluster and initializing the controller with a
default administrator were separate operations.
Now, the raft cluster will be bootstrapped whenever the controller is initialized.
The controller can be initialized as follows:
- Using
ziti agent controller init
- Using
ziti agent controller init-from-db
- Specifying a
db:
entry in the config file. This is equivalent to usingziti agent controller init-from-db
.
Additionally:
minClusterSize
has been removed. The cluster will always be initialized with a size of 1.bootstrapMembers
has been renamed toinitialMembers
. IfinitialMembers
are specified,
the bootstrapping controller will attempt to add them after bootstrap has been complete. If
they are invalid they will be ignored. If they can't be reached (because they're not running
yet), the controller will continue to retry until they are reached, or it is restarted.
Connect Events
These are events generated when a successful connection is made to a controller, from any of:
- Identity, using the REST API
- Router
- Controller (peer in an HA cluster)
They are also generated when an SDK connects to a router.
Controller Configuration
events:
jsonLogger:
subscriptions:
- type: connect
handler:
type: file
format: json
path: /tmp/ziti-events.log
Router Configuration
connectEvents:
# defaults to true.
# If set to false, minimal information about which identities are connected will still be
# sent to the controller, so the `edgeRouterConnectionStatus` field can be populated,
# but connect events will not be generated.
enabled: true
# The interval at which connect information will be batched up and sent to the controller.
# Shorter intervals will improve data resolution on the controller. Longer intervals could
# more efficient.
batchInterval: 3s
# The router will also periodically sent the full state to the controller, to ensure that
# it's in sync. It will do this automatically if the router gets disconnected from the
# controller, or if the router is unable to send a connect events messages to the controller.
# This controls how often the full state will be sent under ordinairy conditions
fullSyncInterval: 5m
# If enabled is set to true, the router will collect connect events and send them out
# at the configured batch interval. If there are a huge number of connecting identities
# or if the router is disconnected from the controller for a time, it may be unable to
# send events. In order to prevent queued events from exhausting memory, a maximum
# queue size is configured.
# Default value 100,000
maxQueuedEvents: 100000
Example Events
{
"namespace": "connect",
"src_type": "identity",
"src_id": "ji2Rt8KJ4",
"src_addr": "127.0.0.1:59336",
"dst_id": "ctrl_client",
"dst_addr": "localhost:1280/edge/management/v1/edge-routers/2L7NeVuGBU",
"timestamp": "2024-10-02T12:17:39.501821249-04:00"
}
{
"namespace": "connect",
"src_type": "router",
"src_id": "2L7NeVuGBU",
"src_addr": "127.0.0.1:42702",
"dst_id": "ctrl_client",
"dst_addr": "127.0.0.1:6262",
"timestamp": "2024-10-02T12:17:40.529865849-04:00"
}
{
"namespace": "connect",
"src_type": "peer",
"src_id": "ctrl2",
"src_addr": "127.0.0.1:40056",
"dst_id": "ctrl1",
"dst_addr": "127.0.0.1:6262",
"timestamp": "2024-10-02T12:37:04.490859197-04:00"
}
SDK Events
Building off of the connect events, there are events generated when an identity/sdk comes online or goes offline.
events:
jsonLogger:
subscriptions:
- type: sdk
handler:
type: file
format: json
path: /tmp/ziti-events.log
{
"namespace": "sdk",
"event_type" : "sdk-online",
"identity_id": "ji2Rt8KJ4",
"timestamp": "2024-10-02T12:17:39.501821249-04:00"
}
{
"namespace": "sdk",
"event_type" : "sdk-status-unknown",
"identity_id": "ji2Rt8KJ4",
"timestamp": "2024-10-02T12:17:40.501821249-04:00"
}
{
"namespace": "sdk",
"event_type" : "sdk-offline",
"identity_id": "ji2Rt8KJ4",
"timestamp": "2024-10-02T12:17:41.501821249-04:00"
}
Component Updates and Bug Fixes
-
github.com/openziti/agent: v1.0.18 -> v1.0.19
-
github.com/openziti/channel/v3: v3.0.5 -> v3.0.10
-
github.com/openziti/edge-api: v0.26.32 -> v0.26.35
-
github.com/openziti/foundation/v2: v2.0.49 -> v2.0.50
-
github.com/openziti/identity: v1.0.85 -> v1.0.88
-
github.com/openziti/metrics: v1.2.58 -> v1.2.59
-
github.com/openziti/runzmd: v1.0.51 -> v1.0.53
-
github.com/openziti/sdk-golang: v0.23.43 -> v0.23.44
-
github.com/openziti/transport/v2: v2.0.146 -> v2.0.150
-
github.com/openziti/ziti: v1.1.15 -> v1.2.0
- Issue #2212 - Rework distributed control bootstrap mechanism
- Issue #1835 - Add access log for rest and router connections
- Issue #2234 - Emit an event when hasEdgeRouterConnection state changes for an Identity
- Issue #2506 - Identity service config overrides referential integrity issues
- Issue #2491 - fix router CSR loading
- Issue #2478 - JWT signer secondary auth: not enough information to continue
- Issue #2482 - router run command - improperly binds 127.0.0.1:53/udp when tunnel mode is not tproxy
- Issue #2474 - Enable Ext JWT Enrollment/Generic Trust Bootstrapping
- Issue #2471 - Service Access for Legacy SDKs in HA does not work
- Issue #2468 - enrollment signing cert is not properly identified
v1.1.15
Release 1.1.15
What's New
- Bug fixes, enhancements and continuing progress on controller HA
Component Updates and Bug Fixes
-
github.com/openziti/channel/v3: v3.0.3 -> v3.0.5
- Issue #146 - Transport options aren't being set in dialer
- Issue #144 - Add ReadAdapter utility
-
github.com/openziti/edge-api: v0.26.31 -> v0.26.32
-
github.com/openziti/sdk-golang: v0.23.42 -> v0.23.43
- Issue #629 - JWT session refresh interprets expiration date incorrectly
-
github.com/openziti/secretstream: v0.1.24 -> v0.1.25
-
github.com/openziti/ziti: v1.1.14 -> v1.1.15
- Issue #2460 - Panic on JWT token refresh
v1.1.14
Release 1.1.14
NOTE: Will not be promoted, as it contains a bug in the ALPN code, fixed in 1.1.15
What's New
- Bug fixes, enhancements and continuing progress on controller HA
Component Updates and Bug Fixes
-
github.com/openziti/edge-api: v0.26.30 -> v0.26.31
-
github.com/openziti/jwks: v1.0.5 -> v1.0.6
-
github.com/openziti/ziti: v1.1.13 -> v1.1.14
- Issue #2119 - Add authentication events
- Issue #2424 - Enabling any health check causes WARNING to be logged
- Issue #2454 - Fix release archive
- Issue #1479 - ziti edge list ... show paginated output but no suggestions on how to go to next page
- Issue #1420 - ziti-cli comma+space causes unhelpful error
- Issue #2207 - ziti edge login --token -- gets "username and password fields are required"
- Issue #2444 - Change default semantic for policies created from the CLI from AllOf to AnyOf
-
github.com/openziti/xweb/v2: v2.1.2 -> v2.1.3
- Issue #2454 - Fix release archive
- Issue #2429 - Controller configurations without default Edge API binding panics
-
github.com/openziti/ziti: v1.1.12 -> v1.1.13
- Issue #2427 - Add low overhead xgress protocol for DTLS links
- Issue #2422 - Busy first hop links should backpressure to xgress senders
- support using "*" in host.v1/host.v2 allowedAddresses
v1.1.13
Release 1.1.13
NOTE: Will not be promoted, as it contains a bug in the ALPN code, fixed in 1.1.15
What's New
- Bug fixes, enhancements and continuing progress on controller HA
Component Updates and Bug Fixes
- github.com/openziti/xweb/v2: v2.1.2 -> v2.1.3
- Issue #2429 - Controller configurations without default Edge API binding panics
- github.com/openziti/ziti: v1.1.12 -> v1.1.13
- Issue #2427 - Add low overhead xgress protocol for DTLS links
- Issue #2422 - Busy first hop links should backpressure to xgress senders
v1.1.12
Release 1.1.12
WARNING
This release contains a bug which prevents the controller from starting if not all edge-* apis are added to the xweb config. This release will not be promoted.
What's New
- Bug fixes, enhancements and continuing progress on controller HA
- Data corruption Fix
Data Corruption Fix
Previous to version 1.1.12, the controller would not handle changes to the policy type of a service policy.
Specifically if the type was changed from Bind -> Dial, or Dial -> Bind, a set of denormalized data would
be left behind, leaving the permissions with the old policy type.
Example:
- Identity A has Bind access to service B via Bind service policy C.
- The policy type of service policy C is changed from Bind to Dial.
- The service list would now likely show that Identity A has Dial and Bind access to service B, instead of
just Dial access.
Mitigation/Fixing Bad Data
If you encounter this problem, the easiest and safest way to solve the problem is to to delete and recreate
the affected service policy.
If changing policy types is something you do on a regular basis, and can't upgrade to a version with the fix,
you can work around the issue by deleting and recreating policies, instead of updating them.
If you're not sure if you have ever changed a policy type, there is a database integrity check tool which can
be run which looks for data integrity errors. It is run against a running system.
Start the check using:
ziti fabric db start-check-integrity
This kicks off the operation in the background. The status of the check can be seen using:
ziti fabric db check-integrity-status
By default this is a read-only operation. If the read-only run reports errors, it can be run
with the -f
flag, which will have it try to fix errors. The data integrity errors caused
by this bug should all be fixable by the integrity checker.
ziti fabric db start-check-integrity -f
WARNINGS:
- Always make a database snapshot before running the integrity checker:
ziti db fabric snapshot <optional path
- The integrity checker can be very resource intensive, depending on the size of your data model.
It is recommended that you run the integrity checker when the system is otherwise not busy.
Component Updates and Bug Fixes
-
github.com/openziti/agent: v1.0.17 -> v1.0.18
-
github.com/openziti/channel/v3: v2.0.143 -> v3.0.3
- Issue #138 - Allow custom message serialization. Add support for a 'raw' message type.
- Issue #82 - Remove transport.Configuration from UnderlayFactory.Create
-
github.com/openziti/edge-api: v0.26.29 -> v0.26.30
-
github.com/openziti/foundation/v2: v2.0.48 -> v2.0.49
-
github.com/openziti/identity: v1.0.84 -> v1.0.85
-
github.com/openziti/jwks: v1.0.4 -> v1.0.5
- Issue #9 - Using NewKey w/ RSA key results in nil pointer exception
-
github.com/openziti/metrics: v1.2.57 -> v1.2.58
-
github.com/openziti/runzmd: v1.0.50 -> v1.0.51
-
github.com/openziti/sdk-golang: v0.23.40 -> v0.23.42
- Issue #625 - traffic optimization: implement support for receiving multi-part edge payloads
-
github.com/openziti/secretstream: v0.1.21 -> v0.1.24
-
github.com/openziti/storage: v0.3.0 -> v0.3.2
-
github.com/openziti/transport/v2: v2.0.143 -> v2.0.146
- Issue #92 - Implement simple traffic shaper
-
github.com/openziti/xweb/v2: v2.1.1 -> v2.1.2
-
github.com/openziti-incubator/cf: v0.0.3 (new)
-
github.com/openziti/dilithium: v0.3.3 -> v0.3.5
-
github.com/openziti/ziti: v1.1.11 -> v1.1.12
- Issue #2413 - Add db anonymization utility
- Issue #2415 - Fix policy denormalization when service policy type is changed
- Issue #2406 - ziti agent controller snapshot-db exit code is always successful
- Issue #2405 - Investigate Older SDKs Not Enrolling Not Connecting in HA
- Issue #2403 - Fix terminator costing concurrency issue
- Issue #2397 - JWKS endpoints w/ new keys do not get refreshed
- Issue #2390 - Update to github.com/openziti/channel/v3
- Issue #2388 - Remove use of ziti fabric add-identity commands in 004-controller-pki.md
v1.1.11
Release 1.1.11
What's New
- This release updates to Go v1.23
- Updates to the latest version of golangci-lint, to allow it to work with the new version of Go
- Linter fixes to address issues caught by updated linter
v1.1.10
Release 1.1.10
What's New
- Bug fixes, enhancements and continuing progress on controller HA
Component Updates and Bug Fixes
- github.com/openziti/cobra-to-md: v1.0.1 (new)
- github.com/openziti/edge-api: v0.26.25 -> v0.26.29
- github.com/openziti/jwks: v1.0.3 -> v1.0.4
- github.com/openziti/ziti: v1.1.9 -> v1.1.10
- Issue #2374 - Edge Routers Do Not Accept JWTs for
openziti
/native
audiences - Issue #2353 - Update go-jose to avoid CVEs
- Issue #2333 - Give zit agent controller snapshot-db same capabilities as ziti fabric db snapshot
- Issue #2343 - Transferring leadership to another controller fails sometimes
- Issue #2374 - Edge Routers Do Not Accept JWTs for
v1.1.9
Release 1.1.9
What's New
- Bug fixes, enhancements and continuing progress on controller HA
- Includes a performance update (Issue #2340) which should improve
connection ramp times. Previously circuits would start with a relatively low window size and ramp slowly. Circuits
will now start with a large initial window size and scale back if they can't keep up
Component Updates and Bug Fixes
-
github.com/openziti/agent: v1.0.16 -> v1.0.17
-
github.com/openziti/channel/v2: v2.0.136 -> v2.0.143
- Issue #136 - Fix timeout on classic dialer
- Issue #134 - Support the dtls transport
-
github.com/openziti/edge-api: v0.26.23 -> v0.26.25
-
github.com/openziti/foundation/v2: v2.0.47 -> v2.0.48
-
github.com/openziti/identity: v1.0.81 -> v1.0.84
-
github.com/openziti/metrics: v1.2.56 -> v1.2.57
-
github.com/openziti/runzmd: v1.0.49 -> v1.0.50
-
github.com/openziti/sdk-golang: v0.23.39 -> v0.23.40
- Issue #601 - Only send config types on service list if controller version supports it
-
github.com/openziti/transport/v2: v2.0.138 -> v2.0.143
- Issue #85 - Update to latest dtls library
-
github.com/openziti/ziti: v1.1.8 -> v1.1.9
- Issue #2343 - Transferring leadership to another controller fails sometimes
- Issue #2340 - Update xgress defaults
- Issue #2336 - Re-enable optional xgress payload mtu, with message framing
- Issue #2091 - Add
scope
andcliend_id
configuration to ext jwt signers - Issue #2318 - Unable to update appData on existing edge routers using PATCH
- Issue #2281 - Session Certificates Should Return a Chain
- Issue #2285 - routers sometimes report link metrics for closed links
- Issue #2282 - Investigate OIDC TOTP Redirect w/ application/json
- Issue #2279 - Ensure xweb initialized before RAFT
- Issue #2277 - docker controller and router deployments - generate a config by default
- Issue #2154 - HA MFA Enrollment returns 500
- Issue #2159 - API Session in HA return 400