-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add directconnect virtual interface and gateway attachment sources #373
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have some questions for the gateway attachment source.
I appreciate the help in advance 🙏
}, | ||
OutputMapper: virtualInterfaceOutputMapper, | ||
InputMapperSearch: func(ctx context.Context, client *directconnect.Client, scope, query string) (*directconnect.DescribeVirtualInterfacesInput, error) { | ||
if strings.HasPrefix(query, connectionIDPrefix) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this Search implementaion allows you to search by conneciton ID rather than Virtual interface ID, nice. In this case you don't have to do a prefix, the docs comment of Search virtual interfaces by connection ID
is enough here I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I see, let's check: @DavidS-ovm was there a particular reason you suggested a prefix on the search?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was under the impression that there might be more search options than just the connection id, which then needed to be disambiguated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense @tomasdembelli if there isn't another potential search option, then let's skip the prefix
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed redundant prefixes.
bd8221b
Looking good so far |
Pushed this branch to #376 to get a valid test run (as external contributions don't get an AWS token). |
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall very good work. A couple of links that could be added, and some small suggestion around blast propagation but overall it looks really good
// https://docs.aws.amazon.com/directconnect/latest/APIReference/API_DirectConnectGatewayAttachment.html#API_DirectConnectGatewayAttachment_Contents | ||
In: true, | ||
// We can't affect the virtual interface | ||
Out: false, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think in this instance we would want the blast to propagate in both directions. If you imagine a graph which is direct-connect-gateway
-> direct-connect-gateway-attachment
-> directconnect-virtual-interface
If we were to change the gateway, the blast radius wouldn't include the interface, because the outbound blast prop is false. I think realistically if you were to change a direct connect connections it could affect the interfaces and the things that use them couldn't it?
Out: false, | |
Out: true, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Virtual interface <-> gateway relation is defined on the virtual interface source: https://github.com/tomasdembelli/aws-source/blob/main/sources/directconnect/virturl-interface.go#L61
So, changing a gateway would affect the associated virtual interfaces.
My understanding is that the attachment source is just a state source. There is no create or delete method for this source. So, it can only change if the associated sources (virtual interface and gateway) are changed.
It doesn't have a terraform source and a List method, so defining the gateway <-> interface relation on the sources itself seemed more robust to me.
But I see your point, in the graph, maybe it is better so see the gateway to interface connection over their attachment.
In that case we need to remove the relation between virtual interface and gateway on the interface (and soon on the gateway) source.
I can do the change (and as you mentioned in the other relevant comments), I just want to make sure we are on the same page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah you're right, I hadn't noticed the direct relationship. In that case this implementation is good
// This is not clearly documented, but it seems that if the gateway is deleted, the attachment state will change to detaching | ||
In: true, | ||
// We can't affect the direct connect gateway | ||
Out: false, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you could probably argue this in both directions, but it's better to err on the side of true
usually. If I ask ChatGPT it seems to think that changes in an interface could affect the gateway, and therefore we need the blast to propagate through the attachment
ChatGPT Response
In AWS, a Direct Connect Virtual Interface (VIF) and a Direct Connect Gateway (DX Gateway) are separate entities, but changes to a Direct Connect Virtual Interface can indirectly impact the traffic flow and connectivity associated with a Direct Connect Gateway. Let me explain how:-
Routing Configuration: A Direct Connect Virtual Interface is associated with a BGP (Border Gateway Protocol) session that facilitates the exchange of routing information between your on-premises network and your AWS resources. When you make changes to the routing configuration of a Virtual Interface (e.g., adding or removing prefixes), it affects how traffic is routed between your on-premises network and the AWS resources associated with the Virtual Interface.
-
Direct Connect Gateway Association: If your Direct Connect Virtual Interface is associated with a Direct Connect Gateway (via attachment to a Virtual Private Gateway or Transit Gateway), changes in routing may impact the traffic flow to and from the resources in the associated VPCs or Transit Gateways. The Direct Connect Gateway relies on the routing information exchanged through the Virtual Interface.
-
BGP Session Changes: Any changes to the BGP session parameters, such as updating BGP communities or modifying the BGP session properties, can impact how routing decisions are made, which, in turn, can affect the Direct Connect Gateway's connectivity.
It's important to note that changes to a Direct Connect Virtual Interface are more likely to affect the specific routing and connectivity associated with that Virtual Interface rather than directly impacting the overall Direct Connect Gateway itself.
Always exercise caution when making changes to network configurations, and ensure that any modifications align with your overall networking requirements and design. It's recommended to carefully plan and test changes, especially in production environments, to minimize the risk of disruptions.
Out: false, | |
Out: true, |
// +overmind:link ip | ||
item.LinkedItemQueries = append(item.LinkedItemQueries, &sdp.LinkedItemQuery{ | ||
Query: &sdp.Query{ | ||
Type: "ip", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The IP source doesn't support a GET using a CIDR, since it returns details of a single IP address. However the rdap-ip-network
does, and gets details of that network block. Can we change this to do a SEARCH
against that type instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// +overmind:link ip | ||
item.LinkedItemQueries = append(item.LinkedItemQueries, &sdp.LinkedItemQuery{ | ||
Query: &sdp.Query{ | ||
Type: "ip", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As above
Type: "directconnect-direct-connect-gateway-attachment", | ||
Method: sdp.QueryMethod_GET, | ||
// returns a single attachment | ||
// TODO: implement this format on the directconnect-direct-connect-gateway-attachment source for GET |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can remove this TODO right?
// TODO: implement this format on the directconnect-direct-connect-gateway-attachment source for GET |
}, | ||
BlastPropagation: &sdp.BlastPropagation{ | ||
// Changes to the attachment won't affect virtual interface | ||
In: false, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Once again, because the attachment is a kind of "bridging" resource (like an IP often is) I'd argue it's worth having this as true
so that changes to the direct connect gateway propagate through the attachment to the interface
In: false, | |
In: true, |
}, | ||
BlastPropagation: &sdp.BlastPropagation{ | ||
// Changes to the attachment won't affect virtual interface | ||
In: false, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As above
In: false, | |
In: true, |
VirtualInterfaceType: sources.PtrString("private"), | ||
VirtualInterfaceName: sources.PtrString("PrivateVirtualInterface"), | ||
DirectConnectGatewayId: sources.PtrString("cf68415c-f4ae-48f2-87a7-3b52cexample"), | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has a field called VirtualGatewayId
which will refer to https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/directconnect#Client.DescribeVirtualGateways, a type that we will soon create. It's easier to create these links now than to go back and do them later. Since we have the naming convention and we know the CLI command for it: describe-virtual-gateways
we can know that the type will be called directconnect-virtual-gateway
so we can create a link in advance.
Personally I usually try to create test data for every field in the possible return value so it's easy to track which ones I've considered for possible links and which I haven't
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think virtaul interface and the virtual gateway are not directly connected. They are indirectly connected over direct connect gateway. That's why I didn't include it in the links on the virtual interface.
Each VPC has a virtual private gateway that connects to the Direct Connect gateway using a virtual private gateway association. The Direct Connect gateway uses a private virtual interface for the connection to the AWS Direct Connect location. There is an AWS Direct Connect connection from the location to the customer data center.
https://docs.aws.amazon.com/directconnect/latest/UserGuide/direct-connect-gateways-intro.html
What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep that reasoning makes sense. In future I think it'd be worth adding VirtualGatewayId
into the test data, and a comment explaining that reasoning so I know that you've considered it and chosen not to include it rather than potentially missed it
@dylanratcliffe thanks for the detailed comments. I intentionally didn't update the ones related to the blast propagation between interface and gateway over the attachment source. |
@DavidS-ovm For the other error, I've simplified the unique attribute and improved the error message. |
Still very similar, I think the generic e2e test suite is not prepared for a source that has some actual syntax checking. Options:
I think a mix of the first two is the way to go: add flag to disable list testing, change the syntax error to be a NOTFOUND What do you think, @dylanratcliffe ?
|
Yeah I'm with @DavidS-ovm, add a flag to |
Thanks @DavidS-ovm, I've done the recommended changes. |
re-ran the tests, they're passing now. |
// https://docs.aws.amazon.com/directconnect/latest/APIReference/API_DirectConnectGatewayAttachment.html#API_DirectConnectGatewayAttachment_Contents | ||
In: true, | ||
// We can't affect the virtual interface | ||
Out: false, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah you're right, I hadn't noticed the direct relationship. In that case this implementation is good
VirtualInterfaceType: sources.PtrString("private"), | ||
VirtualInterfaceName: sources.PtrString("PrivateVirtualInterface"), | ||
DirectConnectGatewayId: sources.PtrString("cf68415c-f4ae-48f2-87a7-3b52cexample"), | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep that reasoning makes sense. In future I think it'd be worth adding VirtualGatewayId
into the test data, and a comment explaining that reasoning so I know that you've considered it and chosen not to include it rather than potentially missed it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, only thing it needs is for these sources to get added to cmd.go
so they actually get loaded: https://github.com/overmindtech/aws-source/blob/main/cmd/root.go#L236
I've addressed this. |
Related to: #326