Commit Graph

75 Commits

Author SHA1 Message Date
Yash Murty 64e8478b18 Fix log source issue of Public and Service category
Change-Id: I54282df97942c30c413dcdd9a8cd00c22cdd6bc0
Implements: blueprint update-access-processor-log-source
2018-08-16 02:11:48 +00:00
Hajime Kondo 1eb2fe4dc0 Add X-Timestamp header to upload file to swift
In uploading files as object to swift,
Response 400 Bad Request with "Missing X-Timestamp header" message.

To fix above, add "X-Timestamp" to request header.

Change-Id: I9615eedc54403a7dba88e3110b6439f7829b3097
2018-04-27 07:50:25 +00:00
Keiichi Hikita 7444891218 Fix tox settings and relevant bugs
I found problem as follows in current tox.ini

- test-requirements can not be read correctly
- some os level command is not in white_list
- one rst file does not follow rst syntac
- fix lack of gitreview file

Purpose of this review request is fixing above problems.

Change-Id: I1d2b29bbc7cdc506f5187c2ef6ee03fd2b908f58
2018-04-18 09:29:01 +09:00
Keiichi Hikita 737e3ee7e8 fix for upstream repository of slogging 2018-04-06 13:16:14 +09:00
Kazuhiro MIYAHARA b973f522b8 Fix parsing request path with slash delimiters
This patch fixes access_processor to parse request path correctly
when the request path includes query with slash delimiters.

For example, existing access_processor's result of log line
with a request path "/v1/acc/con?prefix=YYYY/MM/DD" is as follows:

    account = "acc"
    container_name = "con"
    object_name = "MM/DD"

It is because, existing access_processor splits query from request paths
*AFTER* splits them into account, container_name, object_name.
This patch changes splitting query timing.
2015-09-04 16:09:34 +09:00
John Dickinson e3a4de56cf filter non-v1 protocols on access log parsing 2015-02-15 16:08:05 -08:00
John Dickinson ddfd6371e1 fixed some time zne plumbing 2015-02-12 15:48:16 -08:00
John Dickinson 620222bdb7 updated how to fetch metdata on container 2015-02-12 15:19:23 -08:00
Daisuke Morita c3312216a2 Output usage data in your own timezone
Before this commit, slogging only outputs time log in UTC. This commit
enables slogging to output time in timezone specified in a config file.
And, slogging outputs time in its system timezone by default from this
commit.
2014-09-16 16:17:00 +09:00
Daisuke Morita d221703677 Usage info in JSON format
Before this commit, Slogging only outputs usage info in CSV file.
To enhance usability, the output format of choice is increased.
2014-09-13 17:40:02 -07:00
Daisuke Morita 58a184ca20 Adapt to Swift 1.13.1
This reflects the recent changes in the swift codebase.
2014-09-01 17:14:11 +09:00
leoh0 ca55f20cb8 Adding missing import
Adding errno
2013-04-17 01:15:46 +09:00
David Goetz 61ed362971 fix unit tests 2013-02-11 10:50:04 -08:00
David Goetz afc8247005 ignore proxy logs with any swift.source 2013-02-08 14:16:16 -08:00
Greg Lange acc5e179aa added try/accept in access log delivery 2012-12-14 19:27:02 +00:00
Stef T 6d8ada8447 Changed the dict to a set, thus removing the need for
the sentinal value. Also included the speed difference
test, but it's in a seperate folder and not part of
the normal test-run (for expediancy's sake)
2012-10-04 21:55:49 +02:00
Stef T 9ce73e2e1e vastly improvee the speed of the access_processor's main process method
by returning dict's instead of lists from IPTools. This turns a very
nasty O(n)*3 into O(1)*3 .. much nicer. All tests pass.
2012-09-26 23:38:18 +02:00
Stef T 3b15a26f1f fix for '-' in datafiles causing explosions, mostly
due to the fact that IpTools tries to check if there
is a range configured and '-' isn't any ip/cidr.

'-' is therefore mapped to 0.0.0.0 if iptools is used
2012-09-22 00:20:44 +02:00
John Dickinson d4eab57ba1 updated to use Application instead of BaseApplication
this reflects the changes in the swift codebase
2012-09-19 22:47:39 -07:00
Stef T 9ad12d2259 fixing the tests, so in the absence of iptools, no explosions. Also exporting the variable CIDR_support incase anywhere else needs to know about iptools or not 2012-09-06 19:18:42 +02:00
Stef T 098545a2cd enable fallback to a non-CIDR version if iptools is not installed 2012-08-22 19:38:33 +02:00
Stef T bdc975a509 Enabling the use of CIDR notation in the access config logs. This will
apply to lb_private_ips, service_ips and service_log_sources.
2012-08-20 17:24:31 -04:00
John Dickinson b1286ddc4d releasing 1.1.8 2012-06-06 21:07:39 +00:00
John Dickinson 3a1ff5ba94 fixed imports 2012-05-22 13:35:16 -05:00
John Dickinson 8faa0a8580 log_common memcache passthrough to internal_proxy 2012-05-22 12:41:31 -05:00
John Dickinson 6f3b21e918 removed MemcacheStub in InternalProxy to allow memcache usage based on config 2012-05-21 14:14:49 -05:00
John Dickinson b47a584b90 removed etag check. that doesn't work 2012-05-15 13:35:16 -05:00
John Dickinson 2b74f59798 added content_type config for uploaded logs 2012-05-15 10:17:54 -05:00
John Dickinson 9dac908f55 added original filename to uploaded log 2012-05-15 09:52:44 -05:00
John Dickinson 2c683bd685 version bump to 1.1.7 2012-05-03 11:41:51 -05:00
John Dickinson f03370721e fixed config parsing for log-stats-collector 2012-04-10 15:25:25 -05:00
John Dickinson bb81b9aef4 version bump to 1.1.6 2012-03-28 23:08:12 -05:00
John Dickinson 7ed5a0d23a access log delivery now uses the common utilities in log_common 2012-03-27 15:52:57 -05:00
John Dickinson 6bd1c93d93 silly conf read fix 2012-03-27 15:27:55 -05:00
John Dickinson 4903e57821 split log common tests out 2012-03-26 10:03:31 -05:00
John Dickinson 235aa72da9 explicit queue join_thread calls 2012-03-26 10:03:31 -05:00
John Dickinson c80e63466d updated tests 2012-03-26 10:03:31 -05:00
John Dickinson 3127f365f9 moved FileBuffer to its own file 2012-03-26 10:03:30 -05:00
John Dickinson fff7dcd04b added common functionality from tools that use log_common 2012-03-26 10:03:30 -05:00
John Dickinson b029ff00b2 changed to bare exception 2012-03-26 08:48:56 -05:00
John Dickinson ae0e7e0d1f refactored child worker error handling to fix hangs 2012-03-26 06:57:16 -05:00
David Goetz 33846a0f38 add changes for swift.source 2012-02-29 19:51:23 -08:00
John Dickinson 9823ad4365 version number bump 2012-02-28 09:43:24 -06:00
John Dickinson 84b9818228 removed cancel_join_thread call 2012-02-24 09:52:31 -06:00
John Dickinson e1f6ae55f8 added multiprocessing queue management 2012-02-23 22:22:57 -06:00
John Dickinson 896ccd60ec Added logging and exception catching 2012-02-22 20:56:13 -06:00
John Dickinson b5db8cea2b bumped version for release 2012-01-17 15:32:46 -06:00
John Dickinson b65caf4af0 fixed typo 2012-01-17 11:19:15 -06:00
John Dickinson 3f631e0830 added "hidden_ips" to config. The IPs will be masked in delivered logs.
fixes github issue #25
2012-01-17 11:15:06 -06:00
John Dickinson 0e787ed8a8 changed error catch to Exception 2012-01-17 10:46:56 -06:00