Skip to content

Commit

Permalink
Promote set of tests to stable (#922)
Browse files Browse the repository at this point in the history
  • Loading branch information
@noursaidi
noursaidi authored Jun 28, 2024
1 parent c445798 commit efe1e47
Show file tree
Hide file tree
Showing 22 changed files with 106 additions and 106 deletions.
1 change: 1 addition & 0 deletions docs/guides/development.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ template `etc/schema_readme_template.md`, and must match (case sensitive) the
value of `$section`, otherwise the schema entry is inserted under the `Other`
section


## Releases

The `bin/upgrade_version` tool updates :
Expand Down
44 changes: 22 additions & 22 deletions docs/specs/sequences/generated.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,38 +32,38 @@ Some caveats:

<!-- START GENERATED, do not edit anything after this line! -->
* [bad_target_family](#bad_target_family-preview): Error handling for badly formed target address family
* [broken_config](#broken_config-beta): Check that the device correctly handles a broken (non-json) config message.
* [config_logging](#config_logging-beta): Check that the device publishes minimum required log entries when receiving config
* [device_config_acked](#device_config_acked-beta): Check that the device MQTT-acknowledges a sent config.
* [broken_config](#broken_config-stable): Check that the device correctly handles a broken (non-json) config message.
* [config_logging](#config_logging-stable): Check that the device publishes minimum required log entries when receiving config
* [device_config_acked](#device_config_acked-stable): Check that the device MQTT-acknowledges a sent config.
* [empty_enumeration](#empty_enumeration-preview): Check enumeration of nothing at all
* [endpoint_connection_error](#endpoint_connection_error-preview): Push endpoint config message to device that results in a connection error.
* [endpoint_connection_retry](#endpoint_connection_retry-preview): Check repeated endpoint with same information gets retried.
* [endpoint_connection_success_alternate](#endpoint_connection_success_alternate-preview): Check connection to an alternate project.
* [endpoint_connection_success_reconnect](#endpoint_connection_success_reconnect-preview): Check a successful reconnect to the same endpoint.
* [endpoint_failure_and_restart](#endpoint_failure_and_restart-preview)
* [endpoint_redirect_and_restart](#endpoint_redirect_and_restart-preview)
* [extra_config](#extra_config-beta): Check that the device correctly handles an extra out-of-schema field
* [extra_config](#extra_config-stable): Check that the device correctly handles an extra out-of-schema field
* [family_ether_addr](#family_ether_addr-preview)
* [family_ipv4_addr](#family_ipv4_addr-preview)
* [family_ipv6_addr](#family_ipv6_addr-preview)
* [feature_enumeration](#feature_enumeration-preview): Check enumeration of device features
* [gateway_proxy_events](#gateway_proxy_events-beta): Check that a gateway proxies pointset events for indicated devices
* [pointset_publish](#pointset_publish-beta): Check that a device publishes pointset events
* [pointset_publish_interval](#pointset_publish_interval-beta): Check handling of sample_rate_sec and sample_limit_sec
* [pointset_remove_point](#pointset_remove_point-beta): Check that pointset state does not report an unconfigured point
* [pointset_request_extraneous](#pointset_request_extraneous-beta): Check error when pointset configuration contains extraneous point
* [state_make_model](#state_make_model-beta): Check that a device publishes correct make and model information in state messages
* [state_software](#state_software-beta): Check that a device publishes correct software information in state messages
* [pointset_publish](#pointset_publish-stable): Check that a device publishes pointset events
* [pointset_publish_interval](#pointset_publish_interval-stable): Check handling of sample_rate_sec and sample_limit_sec
* [pointset_remove_point](#pointset_remove_point-stable): Check that pointset state does not report an unconfigured point
* [pointset_request_extraneous](#pointset_request_extraneous-stable): Check error when pointset configuration contains extraneous point
* [state_make_model](#state_make_model-stable): Check that a device publishes correct make and model information in state messages
* [state_software](#state_software-stable): Check that a device publishes correct software information in state messages
* [system_last_update](#system_last_update-stable): Check that last_update state is correctly set in response to a config update.
* [valid_serial_no](#valid_serial_no-beta)
* [valid_serial_no](#valid_serial_no-stable)

## bad_target_family (PREVIEW)

Error handling for badly formed target address family

1. Test skipped: Not a proxied device

## broken_config (BETA)
## broken_config (STABLE)

Check that the device correctly handles a broken (non-json) config message.

Expand All @@ -87,7 +87,7 @@ Check that the device correctly handles a broken (non-json) config message.
1. Check that log category `system.config.receive` level `DEBUG` not logged
1. Check that log category `system.config.parse` level `DEBUG` not logged

## config_logging (BETA)
## config_logging (STABLE)

Check that the device publishes minimum required log entries when receiving config

Expand All @@ -98,7 +98,7 @@ Check that the device publishes minimum required log entries when receiving conf
1. Wait for log category `system.config.parse` level `DEBUG` to be logged
1. Wait for log category `system.config.apply` level `NOTICE` to be logged

## device_config_acked (BETA)
## device_config_acked (STABLE)

Check that the device MQTT-acknowledges a sent config.

Expand Down Expand Up @@ -238,7 +238,7 @@ Check a successful reconnect to the same endpoint.
* Remove `blobset.blobs._iot_endpoint_config`
1. Wait for endpoint config blobset state not defined

## extra_config (BETA)
## extra_config (STABLE)

Check that the device correctly handles an extra out-of-schema field

Expand Down Expand Up @@ -298,13 +298,13 @@ Check that a gateway proxies pointset events for indicated devices

1. Test skipped: Not a gateway

## pointset_publish (BETA)
## pointset_publish (STABLE)

Check that a device publishes pointset events

1. Wait for receive a pointset event

## pointset_publish_interval (BETA)
## pointset_publish_interval (STABLE)

Check handling of sample_rate_sec and sample_limit_sec

Expand All @@ -319,7 +319,7 @@ Check handling of sample_rate_sec and sample_limit_sec
1. Wait for receive at least 4 pointset events
1. Check that time period between successive pointset events is between 15 and 18 seconds

## pointset_remove_point (BETA)
## pointset_remove_point (STABLE)

Check that pointset state does not report an unconfigured point

Expand All @@ -336,7 +336,7 @@ Check that pointset state does not report an unconfigured point
1. Wait for pointset state matches config
1. Wait for pointset event contains correct points

## pointset_request_extraneous (BETA)
## pointset_request_extraneous (STABLE)

Check error when pointset configuration contains extraneous point

Expand All @@ -355,13 +355,13 @@ Check error when pointset configuration contains extraneous point
1. Wait for pointset state matches config
1. Wait for pointset event contains correct points

## state_make_model (BETA)
## state_make_model (STABLE)

Check that a device publishes correct make and model information in state messages

1. Check that make and model in state matches make in metadata

## state_software (BETA)
## state_software (STABLE)

Check that a device publishes correct software information in state messages

Expand All @@ -380,6 +380,6 @@ Check that last_update state is correctly set in response to a config update.
1. Wait for state last_config matches config timestamp
1. Wait for state update complete

## valid_serial_no (BETA)
## valid_serial_no (STABLE)

1. Wait for received serial number matches
5 changes: 1 addition & 4 deletions etc/schema.out
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
RESULT pass schemas device_state_alpha ALPHA 5/5 Schema validation passed
RESULT pass schemas device_state_beta BETA 5/5 Schema validation passed
RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed
RESULT pass schemas events_pointset_beta BETA 5/5 Schema validation passed
RESULT pass schemas events_pointset_stable STABLE 5/5 Schema validation passed
RESULT pass schemas events_system_alpha ALPHA 5/5 Schema validation passed
RESULT pass schemas events_system_beta BETA 5/5 Schema validation passed
RESULT pass schemas events_system_stable STABLE 5/5 Schema validation passed
RESULT pass schemas state_update_alpha ALPHA 5/5 Schema validation passed
RESULT pass schemas state_update_beta BETA 5/5 Schema validation passed
RESULT pass schemas state_update_stable STABLE 5/5 Schema validation passed
18 changes: 9 additions & 9 deletions etc/schema_itemized.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,23 @@
04 pointset_remove_point
05 too_much_state
06 feature_enumeration
07 valid_serial_no RESULT pass schemas device_state_beta BETA 5/5 Schema validation passed
07 valid_serial_no RESULT pass schemas events_system_beta BETA 5/5 Schema validation passed
07 valid_serial_no RESULT pass schemas state_update_beta BETA 5/5 Schema validation passed
07 valid_serial_no RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed
07 valid_serial_no RESULT pass schemas events_system_stable STABLE 5/5 Schema validation passed
07 valid_serial_no RESULT pass schemas state_update_stable STABLE 5/5 Schema validation passed
08 writeback_success
09 writeback_success
10 writeback_success
11 pointset_publish_interval
12 system_mode_restart
13 config_logging
14 broken_config
15 broken_config RESULT pass schemas device_state_beta BETA 5/5 Schema validation passed
15 broken_config RESULT pass schemas events_system_beta BETA 5/5 Schema validation passed
15 broken_config RESULT pass schemas state_update_beta BETA 5/5 Schema validation passed
15 broken_config RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed
15 broken_config RESULT pass schemas events_system_stable STABLE 5/5 Schema validation passed
15 broken_config RESULT pass schemas state_update_stable STABLE 5/5 Schema validation passed
16 broken_config
17 broken_config RESULT pass schemas device_state_beta BETA 5/5 Schema validation passed
17 broken_config RESULT pass schemas events_system_beta BETA 5/5 Schema validation passed
17 broken_config RESULT pass schemas state_update_beta BETA 5/5 Schema validation passed
17 broken_config RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed
17 broken_config RESULT pass schemas events_system_stable STABLE 5/5 Schema validation passed
17 broken_config RESULT pass schemas state_update_stable STABLE 5/5 Schema validation passed
18 broken_config
19 system_last_update RESULT fail schemas state_update_stable STABLE 5/5 Schema violations found
19 system_last_update RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed
Expand Down
22 changes: 11 additions & 11 deletions etc/sequencer.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,25 @@ RESULT pass enumeration.pointset pointset_enumeration ALPHA 5/5 Sequence complet
RESULT skip gateway bad_target_family PREVIEW 0/0 Not a proxied device
RESULT skip gateway gateway_attach_handling ALPHA 0/0 Not a gateway
RESULT skip gateway gateway_proxy_events BETA 0/0 Not a gateway
RESULT pass pointset pointset_publish BETA 5/5 Sequence complete
RESULT pass pointset pointset_publish_interval BETA 5/5 Sequence complete
RESULT pass pointset pointset_remove_point BETA 5/5 Sequence complete
RESULT pass pointset pointset_request_extraneous BETA 5/5 Sequence complete
RESULT pass system broken_config BETA 5/5 Sequence complete
RESULT pass pointset pointset_publish STABLE 5/5 Sequence complete
RESULT pass pointset pointset_publish_interval STABLE 5/5 Sequence complete
RESULT pass pointset pointset_remove_point STABLE 5/5 Sequence complete
RESULT pass pointset pointset_request_extraneous STABLE 5/5 Sequence complete
RESULT pass system broken_config STABLE 5/5 Sequence complete
CPBLTY pass system broken_config.logging ALPHA 1/1 Capability supported
RESULT pass system config_logging BETA 5/5 Sequence complete
RESULT pass system device_config_acked BETA 5/5 Sequence complete
RESULT pass system extra_config BETA 5/5 Sequence complete
RESULT pass system config_logging STABLE 5/5 Sequence complete
RESULT pass system device_config_acked STABLE 5/5 Sequence complete
RESULT pass system extra_config STABLE 5/5 Sequence complete
RESULT pass system family_ether_addr PREVIEW 5/5 Sequence complete
RESULT pass system family_ipv4_addr PREVIEW 5/5 Sequence complete
RESULT pass system family_ipv6_addr PREVIEW 5/5 Sequence complete
RESULT pass system state_make_model BETA 5/5 Sequence complete
RESULT pass system state_software BETA 5/5 Sequence complete
RESULT pass system state_make_model STABLE 5/5 Sequence complete
RESULT pass system state_software STABLE 5/5 Sequence complete
RESULT pass system system_last_update STABLE 6/6 Sequence complete
CPBLTY pass system system_last_update.subblocks ALPHA 1/1 Capability supported
RESULT pass system system_min_loglevel ALPHA 5/5 Sequence complete
RESULT pass system too_much_state ALPHA 5/5 Sequence complete
RESULT pass system valid_serial_no BETA 5/5 Sequence complete
RESULT pass system valid_serial_no STABLE 5/5 Sequence complete
RESULT pass system.mode system_mode_restart ALPHA 5/5 Sequence complete
RESULT pass writeback writeback_failure ALPHA 5/5 Sequence complete
RESULT pass writeback writeback_invalid ALPHA 5/5 Sequence complete
Expand Down
20 changes: 10 additions & 10 deletions etc/sequencer_local.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,20 @@ RESULT pass enumeration empty_enumeration PREVIEW 5/5 Sequence complete
RESULT pass enumeration.features feature_enumeration PREVIEW 5/5 Sequence complete
RESULT skip gateway bad_target_family PREVIEW 0/0 Not a proxied device
RESULT skip gateway gateway_proxy_events BETA 0/0 Not a gateway
RESULT pass pointset pointset_publish BETA 5/5 Sequence complete
RESULT pass pointset pointset_publish_interval BETA 5/5 Sequence complete
RESULT pass pointset pointset_remove_point BETA 5/5 Sequence complete
RESULT pass pointset pointset_request_extraneous BETA 5/5 Sequence complete
RESULT pass system broken_config BETA 5/5 Sequence complete
RESULT pass pointset pointset_publish STABLE 5/5 Sequence complete
RESULT pass pointset pointset_publish_interval STABLE 5/5 Sequence complete
RESULT pass pointset pointset_remove_point STABLE 5/5 Sequence complete
RESULT pass pointset pointset_request_extraneous STABLE 5/5 Sequence complete
RESULT pass system broken_config STABLE 5/5 Sequence complete
CPBLTY pass system broken_config.logging ALPHA 1/1 Capability supported
RESULT pass system config_logging BETA 5/5 Sequence complete
RESULT fail system device_config_acked BETA 0/5 Timeout waiting for config acked
RESULT pass system extra_config BETA 5/5 Sequence complete
RESULT pass system config_logging STABLE 5/5 Sequence complete
RESULT fail system device_config_acked STABLE 0/5 Timeout waiting for config acked
RESULT pass system extra_config STABLE 5/5 Sequence complete
RESULT pass system family_ether_addr PREVIEW 5/5 Sequence complete
RESULT pass system family_ipv4_addr PREVIEW 5/5 Sequence complete
RESULT pass system family_ipv6_addr PREVIEW 5/5 Sequence complete
RESULT pass system state_make_model BETA 5/5 Sequence complete
RESULT pass system state_make_model STABLE 5/5 Sequence complete
RESULT pass system state_software BETA 5/5 Sequence complete
RESULT pass system system_last_update STABLE 6/6 Sequence complete
CPBLTY pass system system_last_update.subblocks ALPHA 1/1 Capability supported
RESULT pass system valid_serial_no BETA 5/5 Sequence complete
RESULT pass system valid_serial_no STABLE 5/5 Sequence complete
22 changes: 11 additions & 11 deletions etc/sequencer_nostate.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,20 @@ RESULT skip enumeration empty_enumeration PREVIEW 0/0 State testing disabled
RESULT skip enumeration.features feature_enumeration PREVIEW 0/0 State testing disabled
RESULT skip gateway bad_target_family PREVIEW 0/0 Not a proxied device
RESULT skip gateway gateway_proxy_events BETA 0/0 Not a gateway
RESULT pass pointset pointset_publish BETA 5/5 Sequence complete
RESULT pass pointset pointset_publish_interval BETA 5/5 Sequence complete
RESULT skip pointset pointset_remove_point BETA 0/0 State testing disabled
RESULT skip pointset pointset_request_extraneous BETA 0/0 State testing disabled
RESULT skip system broken_config BETA 0/0 State testing disabled
RESULT pass pointset pointset_publish STABLE 5/5 Sequence complete
RESULT pass pointset pointset_publish_interval STABLE 5/5 Sequence complete
RESULT skip pointset pointset_remove_point STABLE 0/0 State testing disabled
RESULT skip pointset pointset_request_extraneous STABLE 0/0 State testing disabled
RESULT skip system broken_config STABLE 0/0 State testing disabled
CPBLTY skip system broken_config.logging ALPHA 0/0 Never executed
RESULT skip system config_logging BETA 0/0 State testing disabled
RESULT skip system device_config_acked BETA 0/0 State testing disabled
RESULT skip system extra_config BETA 0/0 State testing disabled
RESULT skip system config_logging STABLE 0/0 State testing disabled
RESULT skip system device_config_acked STABLE 0/0 State testing disabled
RESULT skip system extra_config STABLE 0/0 State testing disabled
RESULT skip system family_ether_addr PREVIEW 0/0 State testing disabled
RESULT skip system family_ipv4_addr PREVIEW 0/0 State testing disabled
RESULT skip system family_ipv6_addr PREVIEW 0/0 State testing disabled
RESULT skip system state_make_model BETA 0/0 State testing disabled
RESULT skip system state_software BETA 0/0 State testing disabled
RESULT skip system state_make_model STABLE 0/0 State testing disabled
RESULT skip system state_software STABLE 0/0 State testing disabled
RESULT skip system system_last_update STABLE 0/0 State testing disabled
CPBLTY skip system system_last_update.subblocks ALPHA 0/0 Never executed
RESULT skip system valid_serial_no BETA 0/0 State testing disabled
RESULT skip system valid_serial_no STABLE 0/0 State testing disabled
Loading

0 comments on commit efe1e47

Please sign in to comment.