This project has been unmaintained for a while. CI jobs don't currently
pass. Zuulv3's dashboard has largely replaced the need for this service
as well as it renders the logs for us instead.
If there is a need for this project elsewhere it can be resurrected and
maintained by others, but for now lets retire it and move on.
Depends-On: https://review.opendev.org/753398
Change-Id: I476b14c7cd9c49270f67897206ec7ad90643703b
Installs libmagic on platforms where is not installed by defaut,
to allow development work on them.
Change-Id: Ia9f7319c375c2dce7dd86c271c731b44a70ec036
Fixes problem where some lines, like pip requirements were not wrapped
because they didn contain any white-space.
This prevents the need to use horizontal browser scrolling, even
on huge screens.
Change-Id: Ia8e32195fc8d9ca5e171506f03099ebc03859bd1
The facility to HTMLify a file from the command line has been
broken since the severity filter was added. This gets it working
again.
Change-Id: I3cb0894014bfbf0e376b8bb60c8da738d93aaffb
Previously we attempted to check supports sev by running re.search
against the full log path of the log file. This meannt that the re that
roughly said: r'/neutron-' could match job logs like this:
http://logs.openstack.org/17/553617/16/check/neutron-tempest-plugin-dvr-multinode-scenario/42fb158/job-output.txt.gz
Unfortunately that meant that all neutron logs had to support sev and if
they didn't (which job output doesn't) then we'd serve no logs to the
user if they set a log level.
We address this by switching to matching on the basename of the path and
by switching to re.match so that we only match from the beginning of the
re.
Change-Id: Ia428dc7339f06a15c58001e99ca677cb6059186b
Since Pike the placement-api logs show up under
screen-placement-api since placement-api is run
using uwsgi rather than Apache.
Change-Id: Ie5428066e6c185de2bd25e2e84823393c6b87d02
Adding the logs of the dragonflow project
(https://github.com/openstack/dragonflow) to the list of logs that
support filtering by severity, to allow better log analyzing for the
dragonflow project.
Change-Id: Id71c25e1987c589cdcd2fc226cfc4755c17cf7dd
Previous commit 9dff1f2af9 added support
for showing loglevel CRITICAL. And now when clicking a loglevel like
ERROR you will see both ERROR and CRITICAL.
But the previous patch forget to add a clickable link for just
CRITICAL messages. This patch adds that.
Change-Id: I472db26dc9f21e0205dbfd0e21c75037d71a07b8
"CRITICAL" is a valid Python log level [1]
We should support it or otherwise if we select to display the "ERROR"
log level we won't see the "CRITICAL" log level messages.
[1] https://docs.python.org/2.7/library/logging.html#logging-levels
Change-Id: Icdf8792dfa605c17f6186b53032ed103b3bfdd79
There are examples of logs from cinder volume at least where we get
lines without severity belonging to the last line with severity.
Properly associate these lines with the last not NONE severity.
Change-Id: I154ac99fe02de7f035c36e18603496f954aac961
The regex ended up removing a space between date and host in systemd
log processing, this puts it back.
Change-Id: If0516e1187422cc76f2b3eaac82280504f94cec0
There was a bug in systemd parsing with a missing space so that it was
never matching. That's fixed.
In addition, the systemd and syslog line parsing are close enough to
each other, that in our previous patterns they would match each
other's lines. The important distinction is that systemd includes
[pid] after the host which syslog doesn't.
We also have to realize that status might not be in a systemd line,
and that's ok, but still match it, given that otherwise we run the
risk of falling into the syslog match.
Change-Id: Id91c167118894c86b9b0d875a1bb36bdbe239ebd
The swift support was never moved into production for infra because of
an unsolved issue around index files, so the logs in swift approach
was ended. This takes out all the swift supporting code to drop the
complexity of the code base. As this is very intermittently maintained
tool, less code makes any future work easier to do.
Change-Id: Iabc4cffc35633fd808556ebb68a8bdd5a5d1fde0
Services supported by the new lib/neutron devstack library use neutron-
prefix not q-. Support both for the time being. Eventually we will get
rid of q- prefix.
The patch also adds neutron-l3 sample file to test suite, to make sure
we don't regress in the future.
Change-Id: I48ca66421d39a8fdd8700adf691823e20a03ab30
Now it can distinguish between random strings and actual log messages
(with timestamps and all). The patch also fixes some cases where levels
were incorrectly classified depending on the order of messages parsed.
The fix triggers some renumbering in calculated dicts with number of
matching messages per level.
The test is still a bit dumb because it does not apply elaborate
classification filtering as the actual middleware does, but that's
enough for now.
The patch is needed for the next patch that will add neutron-l3 log
file.
Change-Id: I0232c416a4dbcafb6741a40eeb75a81037cbf0c8
While ironic itself uses ir- prefix, ironic-inspector used ironic-.
This change will allow it to have better-looking logs in gate.
Change-Id: Id692f64c7fcf2e7c70c264e9a4e6f0d38d48a311
Currently, files with extension .log.gz are not exempt from the
passthrough filter and will be downloaded by the browser instead
of formatted for viewing in the browser. Since some log files in
stable releases use .log for the extension, it's useful to
whitelist these files when served up for log analyzation.
Change-Id: I3d391167bbc3e3b8a4fffd368a0a16f944f548ca
This is primarily needed by our log_processor gearman workers which currently
attempt to retry failed downloads with partial GETs.
Change-Id: Ic49579d253ea773651e4b9641223c2d2ae961c94
It is valuable when looking for the most recent test run to know the mod
time of log files. Add this to the generated index files to provide that
information to our users.
Change-Id: I03669afd73d1f0ce14adce17a3136ce26377b403
File sizes are an important piece of information in log file indexes as
they help readers pick out inappropriately sized files which may
indicate the bug can be found there. Add file sizes to the index so that
we don't lose this info when compared to mod_autoindex.
Change-Id: I248a4fc9bd5ff3fa536e30bcc981a2d5f640307c
When we generate indexes we query disk and swift, if any files have
overlapping paths between them we were generating duplicate index
entries for the same paths. This happened because we used lists to store
the entries which allow for duplicates.
Fix this by using sets until we need to sort (sets are unsorted so we
have to have a list at that point). This will remove any duplicates and
make the index pages more correct.
Change-Id: I6dfa3b30819d6633c3e483d3a386bdce3e26b572
In some cases (such as the new coverage report) there is leading
whitespace before the <html> tag. Skip these lines when trying to
detect if we should escape any content.
This makes os-loganlayze lossy whereby it'll ignore leading whitespace
in html files.
Change-Id: Ie7b183dfa78f58f0b39339ba41850393b248b872
The key thing I wanted to update was the stale pbr req, but I figured
I'd just run update-requirements and take what it made :).
Change-Id: I65e2556a19d81defb4ee6ceb31a0bec6635efe55
If a file isn't found, instead list an index with everything within
that folder from both disk and swift.
Change-Id: I60ef6c625904de6af098df1906598ef107f74796
We can specify via config that a file should be served as html but we
should still only do so if the client supports html. Otherwise fallback
on text. This allows us to have a default for log files that is rendered
html in a browser but text when wgetted or viewed in a text editor.
This corrects an issue where we are attempting to index a bunch of html
into elasticsearch because we always get html for the log files even
though the logstash http client doesn't support html.
Change-Id: I35d78498034a9303bd940f57ac34f423673448db
Because we try swift and then fall back to disk or alternative
path names we get a lot of 404's when looking for a log. These
are currently filling up apache's error.log from our catch-all
exception handler.
We still want to catch all exceptions from attempting swift (as
these will range from 403's through to the swiftclient not existing)
but we want to ignore errors from 404's as they are handled
separately.
Change-Id: I7c03685dfde12a3c09f9a2a9caca811281779875
Match filters and views to files based off their filename. If no
match for a file is found the default set in wsgi.conf will be used
or else a best (legacy) guess is made.
Change-Id: Idac2849b52471acb06eb43b64edb1762a74c6fe1
Currently os-loganalyze will detect what filter and view to use based
off the file type, name and other headers. Instead, allow the config
to define exactly what view to use. If none is set then the legacy
detection will still be applied.
Next change will allow the config to define filters and views based
off file match conditions.
Change-Id: I8955577c100b13ce20609426025a68fbbd052423