Logs that are a “debug diary” are not useless. Their purpose is to debug. That’s why there’s log levels. If you are not interested in that, filter by log levels above debug.
Also, the different formats for fields I see as a necessary evil. Generally, more logs (of verbose log levels) = more good. Which means that there should be as frictionless to write as possible. Forcing a specific format just means that there will be less logs being written.
The json (or any other consistent format) logs seem to be a good idea, but I would keep it to a single debug level (maybe info+error?). So if you want to get wide events, you filter by these log levels to get the full compact picture. But if you are following a debug log chain, it seems a pain to have to search for the “message” field on a potentially order-independent format instead of just reading the log.
TL;DR
Log levels have different purposes, and so they should have different requirements.
LLM would be great to parse all that data, but I think you miss OP’s point. AI can be useful to automate mundane jobs, i.e. jobs you can’t get away from. OP’s point in my view is verbose logs are noisey & difficult to parse, because you’re logging everything unnecessarily. If you Log interesting things and mark them with context & logging levels, Then you can dive in as deep as you need, when you need. Why add complexity (& other hazards) of AI when you can fix the root of the problem first yourself.
Generally agree. Except:
Logs that are a “debug diary” are not useless. Their purpose is to debug. That’s why there’s log levels. If you are not interested in that, filter by log levels above debug.
Also, the different formats for fields I see as a necessary evil. Generally, more logs (of verbose log levels) = more good. Which means that there should be as frictionless to write as possible. Forcing a specific format just means that there will be less logs being written.
The json (or any other consistent format) logs seem to be a good idea, but I would keep it to a single debug level (maybe info+error?). So if you want to get wide events, you filter by these log levels to get the full compact picture. But if you are following a debug log chain, it seems a pain to have to search for the “message” field on a potentially order-independent format instead of just reading the log.
TL;DR
Log levels have different purposes, and so they should have different requirements.
Might be a stupid question but wouldn’t it be a good job for fucking AI to read these and tell you where the interesting parts are?
LLM would be great to parse all that data, but I think you miss OP’s point. AI can be useful to automate mundane jobs, i.e. jobs you can’t get away from. OP’s point in my view is verbose logs are noisey & difficult to parse, because you’re logging everything unnecessarily. If you Log interesting things and mark them with context & logging levels, Then you can dive in as deep as you need, when you need. Why add complexity (& other hazards) of AI when you can fix the root of the problem first yourself.