Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to use from kafka msg? #3

Closed
w2n1ck opened this issue Sep 7, 2021 · 6 comments
Closed

how to use from kafka msg? #3

w2n1ck opened this issue Sep 7, 2021 · 6 comments

Comments

@w2n1ck
Copy link

w2n1ck commented Sep 7, 2021

consume kafka msg, not audit.log file.

@hillu
Copy link
Collaborator

hillu commented Sep 7, 2021

I fail to grasp what you'd like to do. Please elaborate.

@w2n1ck
Copy link
Author

w2n1ck commented Sep 8, 2021

I have written the data of audit.log to Kafka. How do I convert audit.log data from Kafka to Laurel JSON?

@hillu
Copy link
Collaborator

hillu commented Sep 8, 2021

There are two things to consider here

  1. Laurel just reads its data directly from auditd, via standard input. It is intended to be run on the machine where the audit log is produced because it enriches records by accessing information from /proc.
  2. The data that auditd passes to laurel is very similar to the audit log, with one exception: Events originating from the kernel end with an EOE ("end of event") record which is used by the algorithm that coalesces messages belonging to one event. This bit is not written to disk (and probably not written to your Kafka streams.)

While I don't think that it makes a lot of sense to pipe your data from Kafka into Laurel, I think that centralized post-processing is an interesting use-case.

@w2n1ck
Copy link
Author

w2n1ck commented Sep 8, 2021

I have collected all the audit.log on the IDC machine of our company through Filebeat to Kafka and analyzed the all audit.log using ELK. However, since the original audit.log was not intuitive and friendly, the data formatted through Laurel looked friendly and convenient for log analysis

@w2n1ck
Copy link
Author

w2n1ck commented Sep 10, 2021

have any idea or plan ?

@AndyXan
Copy link
Collaborator

AndyXan commented Sep 10, 2021

I dont't think thats the usecase for LAUREL. It's use to generate a SIEM friendlier version of auditd.logs. You can collect those logs itself with filebeat I think. Or otherwise, use auditbeats if you're using ELK already?

Just to clarify, LAUREL is not a "input auditd.logs and output SIEM-friendly logs"-tool.

@w2n1ck w2n1ck closed this as completed Sep 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants