Update the access_processor sanitize_ips function to handle log source and prevent duplicate counts.
Update the access_processor sanitize_ips function to handle log source appropriately and prevent duplicate counts in the slogging file.
Purpose :
This Blueprint tackles the issue of getting duplicate counts in our POST and COPY requests.
Summary :
Currently the access_processor does not distinguish between "Service" and "Public" logs while creating the usage counts in keylist_mapping.
We can fix this by simply updating the sanitize_ips function to not include the "log_source" value for sanitation.
For logs created by plugins in the proxy pipeline, the log_source has some distinct values like "RL", "SSC", etc. It directly maps to the "source" key in the proxy logs definition :-
> The “source” of the request. This may be set for requests that are generated in order to fulfill client requests, e.g. bulk uploads.
https:/
For requests made by clients, the proxy logs show this value to be "-". But the sanitize function will then replace this to be "0.0.0.0" and it will wrongly pass the check in process function and always fall in "Service" category :-
https:/
Hence, the proposal to update the sanitize function to ignore the "log_source" value.
WIP - Fix log source issue of Public and Service category
https:/
Blueprint information
- Status:
- Complete
- Approver:
- Keiichi Hikita
- Priority:
- Medium
- Drafter:
- Yash Murty
- Direction:
- Approved
- Assignee:
- Yash Murty
- Definition:
- Approved
- Series goal:
- None
- Implementation:
-
Implemented
- Milestone target:
- None
- Started by
- Yash Murty
- Completed by
- Yash Murty
Related branches
Related bugs
Sprints
Whiteboard
I and storyboard team is now planning to migrate this launchpad project to storyboard at 31st August.
So if you need to modify this(launchpad) page, please finish it until end of that day.
After 1st of Sep, Storyboard project will newly be created.
Gerrit topic: https:/
Addressed by: https:/
Implements: blueprint update-
Work Items
Work items:
Submit the PR : DONE
