element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • Members
    Members
    • Benefits of Membership
    • Achievement Levels
    • Members Area
    • Personal Blogs
    • Feedback and Support
    • What's New on element14
  • Learn
    Learn
    • Learning Center
    • eBooks
    • STEM Academy
    • Webinars, Training and Events
    • More
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • More
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • More
  • Products
    Products
    • Arduino
    • Dev Tools
    • Manufacturers
    • Raspberry Pi
    • RoadTests & Reviews
    • Avnet Boards Community
    • More
  • Store
    Store
    • Visit Your Store
    • Choose Another Store
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
Avnet Boards Forums
  • Products
  • Dev Tools
  • Avnet Boards Community
  • Avnet Boards Forums
  • More
  • Cancel
Avnet Boards Forums
MicroZed Hardware Design Structure Needs Cleaning Errors
  • Forums
  • Documents
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
Avnet Boards Forums requires membership for participation - click to join
Actions
  • Share
  • More
  • Cancel
Forum Thread Details
  • Replies 9 replies
  • Subscribers 44 subscribers
  • Views 470 views
  • Users 0 members are here
  • microzed
Related

Structure Needs Cleaning Errors

kvikrama
kvikrama 8 months ago

Hi All,

 

   We are using microZED SOM along with AVNET FMC carrier card in one of our industrial products.

We are running a linux based system. In lab conditions the setup works fine. In industrial environment we are experiencing "Structure Needs Cleaning" errors on linux libraries.

This forces us to keep re-flashing the SDCards. Have been looking for source of this problem for sometime.

SOM is Industrial grade (AES-Z7MB-7Z020-SOM-I-G/REV-H) .

 

Notes (Not sure if these can cause issues) :-

    1. The board takes a PMOD CAN module to communicate.

    2. DDR3 Chip Surface Temperature is 55 deg C.

 

A similar setup is working well in same environment just few feet away (another machine).

 

Some of the suspects that we tested and eliminated (might need revisit if any of you advice).

1. We suspected power supply noise and done many checks and filters to remove them. Looks like power supply noise is not the reason.

2. We suspected the linux kernel / application and did many regressions in our lab setup. All of then came clean.

3. We suspected quality of SDCard and moved to industrial SDCard. Didnt resolve the issue.

4. Suspected thermal issues and reduced the temperature on chip surfaces a lot (using RPi heatsink on Zynq 7020 and reducing the CPU speed to half). Didnt help either.

    On DDR3 we havent added heatsink as it's almost always 55 deg C on all our installations.

5. Thought machine grounding can be a problem and wired PMOD CAN ground to machine ground (In addition to grounding the FMC card's mounting holes). Didnt help.

 

Any help thought will be of great help.

  • Reply
  • Cancel
  • Cancel
Parents
  • bhfletcher
    bhfletcher 8 months ago

    We've seen issues with SD Cards, especially generic cheap ones but even some name-brand. I'd be curious to know exactly what you are using. We have had very good success with Delkin Utility-class cards. To see why we use them, check out this blog:

    Storage Insights #1 - Why Not All SD Cards are Created Equal

     

    You should also review this Xilinx Answer Record.

    https://support.xilinx.com/s/article/66779

     

    Is there any chance that power can be interrupted during operation? That can do very bad things to SD cards when Linux isn't shut down properly.

     

    If you do a simple web search, you will find a wide number of answers that are all over the board. Whether they apply to you depends on what you are doing:

    • improper dismounts
      • For example, If the partition you want to unmount has samba shares on it, you need to stop the smbd service first.
    • access from a foreign OS (ie: Windows driver)
    • disk (pre)failure
    • main memory failure

     

    Bryan

    • Cancel
    • Up 0 Down
    • Reply
    • Cancel
  • kvikrama
    kvikrama 8 months ago in reply to bhfletcher

    Hi Bryan,

     

      We are using Apacer card - AK6.112FHA.00103. We have tried other commercial cards as well, frequency of issue doesn't change.

    • Cancel
    • Up 0 Down
    • Reply
    • Cancel
Reply
  • kvikrama
    kvikrama 8 months ago in reply to bhfletcher

    Hi Bryan,

     

      We are using Apacer card - AK6.112FHA.00103. We have tried other commercial cards as well, frequency of issue doesn't change.

    • Cancel
    • Up 0 Down
    • Reply
    • Cancel
Children
  • bhfletcher
    bhfletcher 8 months ago in reply to kvikrama

    The Apacer AK6.112FHA.00103 card is a 3D TLC, which is going to be less reliable than an MLC card, like the Delkin Utility card.

    https://in.element14.com/delkin-devices/s416apg49-u3000-3/memory-card-microsdhc-mlc-16gb/dp/2806491

     

    I'm not saying that is the reason you see failures. However, it could contribute. It's likely interrupted power or some other improper dismount that is the likely culprit.

     

    Bryan

    • Cancel
    • Up 0 Down
    • Reply
    • Cancel
  • zedhed
    zedhed 8 months ago in reply to bhfletcher

    Hi kvikrama

     

    I tend to agree with bhfletcher that improper shutdown can cause corruption, especially if you are in the middle of a write operation since the underlying Flash technology is going to be erasing a block of data back to 1s before writing the updated block info.  This is well documented for Linux systems and you can find information on steps you can take to protect your filesystem that may help here:

    filesystems - Linux file system corruption due to improper shutdown (fs ext4)? - Unix & Linux Stack Exchange

     

    Of course, spurious resets are by nature an improper shutdown of Linux.  Do you see any such evidence of unexpected resets happening on your system?

     

    Have you tried running some of the filesystem check tools to verify that this is the case and get a better understanding of the nature of the corruption?

    ubuntu - Cannot remove file: "Structure needs cleaning" - Unix & Linux Stack Exchange

     

    Which filesystem are you using on the SD card where you see the reported corruptions?  How frequently are you writing to the SD card?

     

    Best Regards,

     

    -Kevin

    • Cancel
    • Up 0 Down
    • Reply
    • Cancel
  • kvikrama
    kvikrama 8 months ago in reply to zedhed

    Hi Kevin,

     

         Application writes into SDCard is once in few minutes one image file (jpeg image). We are using EXT4 Filesystem.

    The errors are in linux libraries and not in the jpeg image files that are written into. 

     

    I am still not sure if the origin of the errors is SDCard itself or it's the DRAM memory that eventually shows up in SDCard.

    • Cancel
    • Up 0 Down
    • Reply
    • Cancel
  • zedhed
    zedhed 8 months ago in reply to kvikrama

    Hi kvikrama

     

    If you are making known writes for only the JPGs, one thing you could try is to mount a separate EXT4 drive (try a USB thumb drive) using MicroZed USB host port and see if the corruption follows the write operations to the new drive.  That would be a clue that something interrupts the write operations over time of the issue moves to the USB thumb drive.

     

    You can also double check your EXT4 options since you should be able to recover from an improper shutdown with the right filesystem options enabled:

    ubuntu - ext4 filesystems frequently corrupting - Unix & Linux Stack Exchange

     

    Best Regards,

     

    -Kevin

    • Cancel
    • Up 0 Down
    • Reply
    • Cancel
Element14

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2022 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • Facebook
  • Twitter
  • linkedin
  • YouTube