Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STM32 Cache coherence #520

Open
mulcmu opened this issue Nov 22, 2024 · 7 comments
Open

STM32 Cache coherence #520

mulcmu opened this issue Nov 22, 2024 · 7 comments

Comments

@mulcmu
Copy link

mulcmu commented Nov 22, 2024

@PeterChristen577 First off, thanks for contributing the STM32 port. Below problem was encountered running on a H7 cpu.

I got the demo running fine on the STM32F746-Disco board. I imported OpENer successfully into a new STM32F746-Disco TouchGFX template project. Needed one small change for freeRTOS CMSIS_V2 vs _V1. Both these were tested with enip/explorer and wireshark. Explicit and implicit messaging working perfect.

I imported into a STM32H745I-Disco TouchGFX template project. Got everything setup for this board but started getting some random data in the enip/explorer implicit messaging. Wireshark showed a few good packets were transmitted and then one or two malformed packets would be sent. Troubleshooting this lead to what appears to be a cache coherence issue.

In cipioconnection.c SendConnectData(...) I changed the the outgoing message to

static ENIPMessage outgoing_message __attribute__((section(".outgoing_message_hack")));

and stuck this with the non-cached lwip heap memory. This got the sample application running on this board without any malformed io packets (at least for single connection). The cpu frequency was higher on this board and the ETH is MII vs RMII which might be contributing factors. I didn't check the F746 to see if malformed packets were still occurring but just way less frequently.

Any ideas for a more elegant solution than the above hack?

@PeterChristen577
Copy link
Contributor

Hi mulcmu

Same questions/ideas:

  • Which heap size have you configured in FreeRTOSConfig.h?
  • What is the MEM_SIZE in lwiopts.h
  • Did you have a look to the document OpENer STM32 Port.pdf?
    There are some hint for setup of lwiopts.h

Best regards Peter

@PeterChristen577
Copy link
Contributor

... another thought:
increase OPENER_STACK_SIZE in opener.c, is currently at 2000

@mulcmu
Copy link
Author

mulcmu commented Nov 28, 2024

The freeRTOS heap size is #define configTOTAL_HEAP_SIZE ((size_t)0x8000)

lwip heap is 96k. First 32k of D2 sram is used for Ethernet dma descriptors and rx buffers, rest of the 128k is used for lwip.
#define MEM_SIZE 0x18000

The graphics buffers are in the external 16mb SDRAM.

Here's snapshot of the stack sizes. OpENer was already increased to (4*1024) as well as increases to the default lwip tasks.
image

@PeterChristen577
Copy link
Contributor

Everything looks good in the stack snapshot, let me do a similar analysis on the STM32F746 target.
One question: Which cycle time have you set up in enip/explorer? Is the rate of malformed packet related to the cycle time?

@mulcmu
Copy link
Author

mulcmu commented Nov 29, 2024

Cycle time was the default 200ms. At 200ms the malformed packets appeared randomly distributed, I didn't see any patterns.

This post on ST community forum discusses similar problem. Seems that on my H745, sometimes the processor cached data is written out to the memory before the DMA reads it for the valid packets. For the malformed packets the outgoing message buffer data is only in the MCU cache, so the DMA is copying the SRAM memory that holds random data.

EipStatus SendUdpData(const struct sockaddr_in *const address,
const ENIPMessage
*const outgoing_message) {
#if defined(OPENER_TRACE_ENABLED)
static char ip_str[INET_ADDRSTRLEN];
OPENER_TRACE_INFO(
"UDP packet to be sent to: %s:%d\n",
inet_ntop(AF_INET, &address->sin_addr, ip_str, sizeof ip_str),
ntohs(address->sin_port) );
#endif
int sent_length = sendto( g_network_status.udp_io_messaging,
(char *)outgoing_message->message_buffer,
outgoing_message->used_message_length, 0,
(struct sockaddr *) address, sizeof(*address) );
if(sent_length < 0) {
int error_code = GetSocketErrorNumber();
char *error_message = GetErrorMessage(error_code);
OPENER_TRACE_ERR(
"networkhandler: error with sendto in SendUDPData: %d - %s\n",
error_code,
error_message);
FreeErrorMessage(error_message);
return kEipStatusError;
}
if(sent_length != outgoing_message->used_message_length) {
OPENER_TRACE_WARN(
"data length sent_length mismatch; probably not all data was sent in SendUdpData, sent %d of %d\n",
sent_length,
outgoing_message->used_message_length);
return kEipStatusError;
}
return kEipStatusOk;
}

I reverted my prior hack and then on line 693 of the SendUdpData() an explicit cache flush was added.

  SCB_CleanDCache_by_Addr(outgoing_message->message_buffer, sizeof(ENIPMessage));

This change also resolved the malformed packet issue for the implicit messaging.

@PeterChristen577
Copy link
Contributor

Great, that you found out this so quickly!
If I understand correctly the situation this cache flush is needed only for H7 targets and not for F7, isn't it?

@mulcmu
Copy link
Author

mulcmu commented Dec 3, 2024

I was still working through things, I have 2 H7 (H757 & H745), an H563, and the F746. The clean cache() function doesn't seem to hurt the F746 but there were a few times ping, http, and list identity would work but then the rest of the ethernet/IP appeared with errors. Still trying to figure out what was changing. I started a branch with the changes. https://github.com/mulcmu/OpENer/tree/stm32-flush-cache-CMSISv2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants