We all did it, most of us still do it. First we create nice log files, log everything we think is useful and save it to a file in a logical place.
Three months later the file is growing out of bounds, its missing the important informations but is full of useless crap that is not detailed enough to tell anything about whats going on and the place where it is doesn’t make any sense anymore. Why is this happening? Why do we do this again and again, even truncating the log files without stopping the whole process?
Yeah, logging is good. The more data we log the better we can solve problems. Problems we get known of through the complaints from our clients because we log the error but we do not alert anyone. Why is it always the client that has to tell that something is going wrong? Shouldn’t it be the other way round? Shouldn’t we as Service Provider or whatever know whenever one of our services fails and tell our clients about it? I guess that would make the relationship to the clients massively better.
So what would be a better way to all of this? I think its pretty simple. And I hope I solved it for now for my current project.
We already had a Log-Class that we were using in existing projects, I just extended that to make life easier. Now we have a configurable log level, for both logging to a file and logging to a DB. For example we log warnings and above to the file and errors and above to the DB. We add a unique identifier (mainly for the file if there can be multiple accesses and the log messages would be mixed), a message and a log level. The identifier can contain the IP of the server and the path to the script as well to be able to identify the exact location of the problem.
On another server there is running a script as cronjob every 5 minutes. This script checks the DB with the log entries sends an summarized email if needed, marks the existing log entries as done and deletes all data that is older than 24 hours. So this is a very easy way to really know whats going on when it is going on.
Now I wonder why I not always did it like that. Maybe there was this irrational thought in my brain that logging is only needed while there is active development, but of course this is not the case, errors come always.
I hope this will remind me of keeping it like this and not falling back to the old behaviour, one day…