Motivation
Some clients require storing logs for a very long time and being able to search through them. Currently, in production, our logs are stored for about 60 days, which is not much—we would like to have the ability to increase this retention period indefinitely (potentially) and allow users to define the retention period themselves.
Solution
It is necessary to implement the export of logs to an ice storage (ice storage, because we already have a cold storage).
At the first stage, the ice storage will be S3, so we need to learn how to work with S3. Later, other storage solutions may be added (Clickhouse, NFS, etc.). It should also be possible to search through this data.
Motivation
Some clients require storing logs for a very long time and being able to search through them. Currently, in production, our logs are stored for about 60 days, which is not much—we would like to have the ability to increase this retention period indefinitely (potentially) and allow users to define the retention period themselves.
Solution
It is necessary to implement the export of logs to an ice storage (ice storage, because we already have a cold storage).
At the first stage, the ice storage will be S3, so we need to learn how to work with S3. Later, other storage solutions may be added (Clickhouse, NFS, etc.). It should also be possible to search through this data.