[chbot] Volatile struct puzzle
Charles Manning
cdhmanning at gmail.com
Mon Feb 7 02:00:07 GMT 2022
Hi Robin
The packing of the structure should not be a factor unless you are fiddling
with the packed attribute.
When we're talking about caching then there are many things that enter the
picture.
You say that the buffer itself (which I expect is the target of the DMA
rather than anything else) has the correct caching attributes. What are
those? From a brief glimpse at the Cortex M7 manuals, you should be
ensuring this has the shared attribute. If this is set correctly then the
caching should not matter. What have you done to check the caching is
correct?
Do not assume that the caching is correct from compiler attributes. Those
might not match the settings in the MMU.
What you are observing is that in one case the cache appears to be fetched
correctly and in the other not.
This can be caused by execution of code far away from this point due to how
the cache works.
A compiler change can reorder instructions and data accesses(especially at
-O3). THis can completely change the CPU's interaction with the cache.
Throw in an out of order CPU like the M7 and a lot can change.
A cache has multiple cache lines and the address being accessed can only
map to a few of these cache lines (termed a set). If other code elsewhere
needs something in the cache that maps to the same set, then this could be
forcing a new cache read - causing the data to be healthy. If, however, the
cache is not being refreshed, then the old cached value might be used
forever.
Assuming the DMA controller is only modifying the buffer, I would try
adding the following just before accessing the buffer:
/* Force fresh data into the cache */
uint32_t jpeg_base = ((uint32_t) JPEG_Data_Buffer) & (~0x1f); /* Calc base
32-byte boundary */
uint32_t n_bytes = ((uint32_t) JPEG_Data_Buffer) - jpeg_base +
sizeof(JPEG_Data_Buffer);
SCB_InvalidateDCache_by_Addr(jpeg_base, n_bytes);
... now access stuff in the cache.
On Sun, Feb 6, 2022 at 3:18 PM Robin Gilks <gb7ipd at gmail.com> wrote:
> Paul:
> Sorry your comment is too cryptic for this self taught machine code
> programmer.
> It could well be that I should add a few memory barrier instructions but
> none of the ST Micro example code uses them and the only helpful? comment
> in any of the example code is as follows:
> @Note If the user code size exceeds the DTCM-RAM size or starts from
> internal cacheable memories (SRAM1 and SRAM2),that is shared between
> several processors,
> then it is highly recommended to enable the CPU cache and maintain
> its coherence at application level.
> The address and the size of cacheable buffers (shared between CPU
> and other masters) must be properly updated to be aligned to cache line
> size (32 bytes).
>
> @Note It is recommended to enable the cache and maintain its coherence,
> but depending on the use case
> It is also possible to configure the MPU as "Write through", to
> guarantee the write access coherence.
> In that case, the MPU must be configured as Cacheable/Bufferable/Not
> Shareable.
> Even though the user must manage the cache coherence for read
> accesses.
> Please refer to the AN4838 “Managing memory protection unit (MPU) in
> STM32 MCUs”
> Please refer to the AN4839 “Level 1 cache on STM32F7 Series”
>
> As already noted, I've assigned the memory explicitly in the DTCM area and
> made sure it fits!
>
> Charles:
> 1) Processor is stm32f779ii so it has all the fandangles
> 2) I'm using -O3 currently as although I'm topping out the bus bandwidth
> with DMA, the CPU has a lot to do at the same time
> 3) I'm aware that volatile and caching are not related as such but since
> the buffer point and state variable in the stuct are the only things
> changed between compiler versions, and the map file(s) indicates that
> EVERYTHING is in the same place (in the DTCM sections anyway - minor
> changes elsewhere in the code as you would expect with over 1.2m of code) I
> can only assume it is the way the compiler treats a struct element between
> static and run time setting. All bets are off by the way if the struct is
> NOT volatile, as expected as it is referenced by the end-of-DMA callback as
> well as the code that keeps the buffers filled as the JPEG engine works its
> way through a bitmap image.
>
>
>
> On Sun, Feb 6, 2022 at 9:45 AM Charles Manning <cdhmanning at gmail.com>
> wrote:
>
>> There are a few important points missing here:
>> 1) Which STM32 processor? They are not all just "STM32" processors. They
>> range from the very simple Cortex M0 and M0+, through the M3, M4 and M33 up
>> to the M7.
>> Different cores have different caching behaviour. The M0 and M0+ have
>> zero caching and the M7 has extensive caching. If you're having a caching
>> issue then I suppose it is more likely to be an M7. Can you confirm that?
>> 2) What compilation options are you using?
>> 3) Volatile has pretty much nothing to do with caching. All volatile does
>> is ensuring that a registered value is not used and a memory access is
>> forced. It also ensures that the order of volatile accesses is maintained
>> (from a CPU perspective, not a cache perspective). It does nothing to
>> ensure cache coherency. For that you need to do what Paul says: uses memory
>> barriers.
>>
>> Assuming you're using an M7, this is not your granny's microcontroller.
>> It does not perform memory accesses in order. It is superscaler (ie. it can
>> execute multiple instructions in parallel and does not execute them in
>> order), has a sophisticated branch predictor and, most importantly here -
>> it has a large cache. Thus you need to be careful to get your various
>> accesses in order.
>>
>> https://www.sciencedirect.com/topics/engineering/m7-processor
>>
>> http://ww1.microchip.com/downloads/en/DeviceDoc/Managing-Cache-Coherency-on-Cortex-M7-Based-MCUs-DS90003195A.pdf
>>
>> https://www.sciencedirect.com/topics/engineering/memory-barrier-instruction
>>
>> The Microchip doc gives some good explanations. It doesn't really matter
>> if it is an ST - what matters is that the core is the same.
>>
>> You might find the cache controll functions and data barrier instructions
>> are helpful.
>> SCB_CleanDCache_by_Addr (uint32_t *addr, int32_t dsize)
>> __DSB()
>> etc
>>
>> Regards
>>
>> Charles
>>
>>
>>
>>
>> On Sun, Feb 6, 2022 at 1:12 AM Paul Davey <plmdvy at gmail.com> wrote:
>>
>>> Have you tried sprinkling memory barriers between the CPU access to
>>> the DMA memory and the DMA operation trigger?
>>>
>>> On Sat, Feb 5, 2022 at 8:23 PM Andrew Dachs <dachsa492 at gmail.com> wrote:
>>> >
>>> > I see. Could it be that the initialisation code changes are just
>>> moving things about in memory rather than root cause? Why is it important
>>> that the control struct is volatile or is it really just the contents of
>>> DataBuffer that’s being changed?
>>> >
>>> >
>>> >
>>> > Sent from my iPhone
>>> >
>>> > On 5/02/2022, at 7:17 PM, Robin Gilks <gb7ipd at gmail.com> wrote:
>>> >
>>> >
>>> > It is outside .data and explicitly NOT initialised.
>>> > Excerpts from the linker file...
>>> >
>>> > /* Specify the memory areas */
>>> > MEMORY
>>> > {
>>> > DTCM (xrw) : ORIGIN = 0x20000000, LENGTH = 128k
>>> > RAM (xrw) : ORIGIN = 0x20020000, LENGTH = 368K
>>> > BOOTRAM (xrw) : ORIGIN = 0x2007C000, LENGTH = 16K
>>> > FLASH (rx) : ORIGIN = 0x8000000, LENGTH = 2048K
>>> > }
>>> >
>>> > /* used by audio buffers and JPEG h/w */
>>> > .MCU_Data_section 0x20000000 (NOLOAD) : { *(.MCU_Data_section) } >DTCM
>>> >
>>> > The DTCM memory is by default available to all internal DMA buses (so
>>> saves playing with the cache setup using 'HAL_MPU_ConfigRegion()'.
>>> >
>>> >
>>> > On Sat, Feb 5, 2022 at 6:47 PM Andrew Dachs <dachsa492 at gmail.com>
>>> wrote:
>>> >>
>>> >> Hi Robin,
>>> >> Is your special area of memory inside .data and part of the
>>> initialisation on startup?
>>> >> Andy
>>> >>
>>> >>
>>> >>
>>> >> Sent from my iPhone
>>> >>
>>> >> On 5/02/2022, at 5:42 PM, Robin Gilks <gb7ipd at gmail.com> wrote:
>>> >>
>>> >>
>>> >> Hmmm - thanks Mark, I hadn't thought of packing issues.
>>> >> It's not like I'm trying a union to another struct so positioning
>>> would be critical. That being the case I think I'll keep that thought on
>>> the back burner for now..
>>> >>
>>> >> Cheers
>>> >>
>>> >>
>>> >> On Sat, Feb 5, 2022 at 4:10 PM Mark Atherton <markaren1 at xtra.co.nz>
>>> wrote:
>>> >>>
>>> >>> I have seen issues with data segments, where large chunks (u32, u8*)
>>> >>> follow small chunks (u8).
>>> >>>
>>> >>> Try changing
>>> >>>
>>> >>> > uint8_t State;
>>> >>> > uint8_t *DataBuffer;
>>> >>> > uint32_t DataBufferSize;
>>> >>>
>>> >>> to
>>> >>>
>>> >>> > uint32_t DataBufferSize;
>>> >>> > uint8_t *DataBuffer;
>>> >>> > uint8_t State;
>>> >>>
>>> >>> -Mark
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On 5/02/2022 1:50 PM, Robin Gilks wrote:
>>> >>> > Some may recall that a year or 2 ago I had great problems getting
>>> the
>>> >>> > hardware JPEG encode to work on an STM32 processor. It turned out
>>> that
>>> >>> > the memory cache attributes manifested as a race condition between
>>> DMA
>>> >>> > and CPU access.
>>> >>> > The problem has just reappeared as a result of updating
>>> >>> > from arm-none-eabi-gcc-9.2.1-1.1 to arm-none-eabi-gcc-10.3.1-2.3.
>>> >>> >
>>> >>> > After a few pokes at the code I've narrowed the issue down to
>>> whether a
>>> >>> > volatile struct element is initialized statically or at run time.
>>> >>> > Interestingly the issue is inverted between compiler versions
>>> >>> > The buffer referenced by the structure element 'DataBuffer' is
>>> defined
>>> >>> > and instantiated in a reserved section of memory that has the
>>> correct
>>> >>> > cache attributes
>>> >>> >
>>> >>> > typedef struct
>>> >>> > {
>>> >>> > uint8_t State;
>>> >>> > uint8_t *DataBuffer;
>>> >>> > uint32_t DataBufferSize;
>>> >>> > }JPEG_Data_BufferTypeDef;
>>> >>> > uint8_t
>>> >>> >
>>> JPEG_Data_InBuffer[CHUNK_SIZE_IN]__attribute__((section(".MCU_Data_section")));
>>> >>> > original code - works with gcc-9; fails with gcc-10:
>>> >>> > volatile JPEG_Data_BufferTypeDef Jpeg_IN_BufferTab =
>>> {JPEG_BUFFER_EMPTY,
>>> >>> > JPEG_Data_InBuffer, 0};
>>> >>> > current code fails with gcc-9 works with gcc-10:
>>> >>> > volatileJPEG_Data_BufferTypeDefJpeg_IN_BufferTab= {0, 0, 0};
>>> >>> > ...
>>> >>> > Jpeg_IN_BufferTab.DataBufferSize = 0;
>>> >>> > Jpeg_IN_BufferTab.State = JPEG_BUFFER_EMPTY;
>>> >>> > Jpeg_IN_BufferTab.DataBuffer = JPEG_Data_InBuffer;
>>> >>> >
>>> >>> > Note that NO other changes (apart from the compiler version) were
>>> made!!
>>> >>> > I'm at a total loss as to what is going on, hopefully someone can
>>> shine
>>> >>> > a light ;)
>>> >>> >
>>> >>> > --
>>> >>> > Robin Gilks
>>> >>> >
>>> >>> >
>>> >>> > _______________________________________________
>>> >>> > Chchrobotics mailing list Chchrobotics at lists.ourshack.com
>>> >>> > https://lists.ourshack.com/mailman/listinfo/chchrobotics
>>> >>> > Mail Archives: http://lists.ourshack.com/pipermail/chchrobotics/
>>> >>> > Meetings usually 3rd Monday each month. See http://kiwibots.org
>>> for venue, directions and dates.
>>> >>> > When replying, please edit your Subject line to reflect new
>>> subjects.
>>> >>> >
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> Chchrobotics mailing list Chchrobotics at lists.ourshack.com
>>> >>> https://lists.ourshack.com/mailman/listinfo/chchrobotics
>>> >>> Mail Archives: http://lists.ourshack.com/pipermail/chchrobotics/
>>> >>> Meetings usually 3rd Monday each month. See http://kiwibots.org for
>>> venue, directions and dates.
>>> >>> When replying, please edit your Subject line to reflect new subjects.
>>> >>
>>> >> _______________________________________________
>>> >> Chchrobotics mailing list Chchrobotics at lists.ourshack.com
>>> >> https://lists.ourshack.com/mailman/listinfo/chchrobotics
>>> >> Mail Archives: http://lists.ourshack.com/pipermail/chchrobotics/
>>> >> Meetings usually 3rd Monday each month. See http://kiwibots.org for
>>> venue, directions and dates.
>>> >> When replying, please edit your Subject line to reflect new subjects.
>>> >>
>>> >> _______________________________________________
>>> >> Chchrobotics mailing list Chchrobotics at lists.ourshack.com
>>> >> https://lists.ourshack.com/mailman/listinfo/chchrobotics
>>> >> Mail Archives: http://lists.ourshack.com/pipermail/chchrobotics/
>>> >> Meetings usually 3rd Monday each month. See http://kiwibots.org for
>>> venue, directions and dates.
>>> >> When replying, please edit your Subject line to reflect new subjects.
>>> >
>>> > _______________________________________________
>>> > Chchrobotics mailing list Chchrobotics at lists.ourshack.com
>>> > https://lists.ourshack.com/mailman/listinfo/chchrobotics
>>> > Mail Archives: http://lists.ourshack.com/pipermail/chchrobotics/
>>> > Meetings usually 3rd Monday each month. See http://kiwibots.org for
>>> venue, directions and dates.
>>> > When replying, please edit your Subject line to reflect new subjects.
>>> >
>>> > _______________________________________________
>>> > Chchrobotics mailing list Chchrobotics at lists.ourshack.com
>>> > https://lists.ourshack.com/mailman/listinfo/chchrobotics
>>> > Mail Archives: http://lists.ourshack.com/pipermail/chchrobotics/
>>> > Meetings usually 3rd Monday each month. See http://kiwibots.org for
>>> venue, directions and dates.
>>> > When replying, please edit your Subject line to reflect new subjects.
>>>
>>> _______________________________________________
>>> Chchrobotics mailing list Chchrobotics at lists.ourshack.com
>>> https://lists.ourshack.com/mailman/listinfo/chchrobotics
>>> Mail Archives: http://lists.ourshack.com/pipermail/chchrobotics/
>>> Meetings usually 3rd Monday each month. See http://kiwibots.org for
>>> venue, directions and dates.
>>> When replying, please edit your Subject line to reflect new subjects.
>>
>> _______________________________________________
>> Chchrobotics mailing list Chchrobotics at lists.ourshack.com
>> https://lists.ourshack.com/mailman/listinfo/chchrobotics
>> Mail Archives: http://lists.ourshack.com/pipermail/chchrobotics/
>> Meetings usually 3rd Monday each month. See http://kiwibots.org for
>> venue, directions and dates.
>> When replying, please edit your Subject line to reflect new subjects.
>
> _______________________________________________
> Chchrobotics mailing list Chchrobotics at lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/chchrobotics
> Mail Archives: http://lists.ourshack.com/pipermail/chchrobotics/
> Meetings usually 3rd Monday each month. See http://kiwibots.org for
> venue, directions and dates.
> When replying, please edit your Subject line to reflect new subjects.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ourshack.com/pipermail/chchrobotics/attachments/20220207/dc01a18f/attachment-0001.html>
More information about the Chchrobotics
mailing list