Oh Boy, Ive really done it now.
When I told our OEM PLC supplier that we need to delete old log files, They just about lost it. We can write files, every second of everyday, but when the on board memory gets full,
there is no method to remove old datalogs. After a fun couple of weeks of being treated like some sort of terrorist, I have a few ideas. None of them are as solid as simply having the ability to delete a file.
I heard it all. Why would you want to purge old logs?
youll loose data.
maybe you could not log as often.
maybe you could not log all the data...
on an on. .. just becuase there is no delete file functionality.
The standard practice I guess is to send a tech out to remove old files from disk. Our products are autonomous, when, if, they fail, we benefit from concise, 1 second logs. We end up with 16MB files every day. On a 16GB SD Card, the day our SD Cards become full is rapidly approaching.
When I spotted this potential issue, management told me, oh its no issue, the old logs will be overwritten with new logs. I see where the documentation seems to suggest this, but this is a miss interpretation of a specific EEPROM storage device, not the SD Card. In testing, I found new logs are not saved, and old logs remain. So if, when, our devices fail, technicians, and application support specialists will not have current data to go off from.
So Ive been on defcon 1 ever since, trying to garner the supporting mechanisms to monitor, and clean logs from our PLC systems prior to any future event which may occur.
I've got a decent level of understanding for the internal control protocols of our systems. The issue there is that I have no way to implement sending an internal commad, to the PLC from the PLC. I havent given up on the notion, but I have seriously buggered some PLCs while trying to create an implementation of this.
There is no direct documentation, for the SD Cards internal register addressing, except an error that appears on screen for an HMI durring a communication fault, and a rather odd "1:N"
listing in what I can only really assume is an artifact of old code. I think the SD Card Device is 0:0 in the internal PLC device network structure. which comes from snooping on network traffic. Decoding the commands neccisary to perform the operation, then reporting them back to the OEM did not go as hoped.
Aside from simply formulating the commands to perform the delete old log files operation, I would also have to write a directory listing function, string parser, and scripts to process the commands, if I could figure out how to send them from the PLC.None of that was going very well.
Due to some success in creating other logging, and monitoring applications for PLC / PCs, I have been assigned to a variety of logging and monitoring duties for the last several months.
Where my integration of sending PLC system commands is still in the air for being deployable in PLC system code, I decided to consider other ideas.
The most novel one yet, Im calling
"Time Warp". In an effort to understand what systems do when there is a lot of data, Ive made scripts that generate dated log files, by manipulating the system clock on my PC, and repeatedly copying datafiles. This gave me the idea to do the same thing on the PLC.
I can not reliably delete a file yet, and when the SD Card is full, new logs are not kept.
What I can do, is write less data to the same files, by manipulating the system clock.
Here is a concept test, where I have a PLC log file, with 69 data points per line. Then, I alter the system time, when the logs are full. This alters the output path of the data, for a datalog block, which then populates (over writes) with less data, for files named on the basis of the date when they were created.
Is it prone to lots of little problems I wish I didn't have to deal with?
Hell yeah it is! But it will work. Probably, for now.
Whats this even do?
well, when I detect that the SD Card is near full, I turn off the normal logging process.
Then, I save the current date and time.
on the basis of which code base is known to log at a given rate, I formulated the amount of data / time which the system must have operated, and define a set of start and end log maintenance times.
Next, I set the time to 2 years ago, and start pulsing the system time seconds register 1000 times a second. This steps through the log files with a 1 bit write, which is significantly less data than we normally log. I removed the time stamp after initial testing.
So then, I rapidly increment though time, overwriting files line by line, with 1 byte per line.
Where Im manipulating the system clock via the seconds register, the PLC manages the SD Card file IO / file names for me.
then where I reach the desired cut off time, I stop the log over write process, reset the system time, and resume normal logging.
CRAZY.
What happens in 10 years, when the SD Card is full of files with 86400 1s?
IDK, maybe I'll have convinced an PLC OEM that deleting a log file is a valid form of system maintenance by then.