Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3 - support for different log types, performance & other improvements #250

Merged
merged 76 commits into from
Aug 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
58f7763
WIP: http access logs
arukompas Jul 10, 2023
90daf6d
Fix styling
arukompas Jul 10, 2023
f165329
WIP: http access & error logs, with more tests
arukompas Jul 11, 2023
a4c64b9
Merge branch 'access-logs' of github.com:opcodesio/log-viewer into ac…
arukompas Jul 11, 2023
9c9dce8
Fix styling
arukompas Jul 11, 2023
37c82f6
more improvements, bug fixes
arukompas Jul 11, 2023
c0e98c3
more improvements, support Nginx error logs
arukompas Jul 12, 2023
6ad4ef2
Fix styling
arukompas Jul 12, 2023
f7c25af
WIP: displaying access logs in the UI
arukompas Jul 13, 2023
5a0a05b
Merge branch 'access-logs' of github.com:opcodesio/log-viewer into ac…
arukompas Jul 13, 2023
83ab161
Fix styling
arukompas Jul 13, 2023
89b4ed9
support for large access log files, other improvements
arukompas Jul 13, 2023
6540c72
small improvements
arukompas Jul 14, 2023
02ffb71
Fix styling
arukompas Jul 14, 2023
1d0d640
refactor LogReader to be more flexible towards more log types
arukompas Jul 14, 2023
c4f9909
Merge branch 'access-logs' of github.com:opcodesio/log-viewer into ac…
arukompas Jul 14, 2023
c99deec
Fix styling
arukompas Jul 14, 2023
1569d39
remove unused HttpLogReader and fix unit tests for the LogReader
arukompas Jul 14, 2023
460c7b6
speed improvements
arukompas Jul 15, 2023
df1af64
Fix styling
arukompas Jul 15, 2023
7273c23
lots of improvements, refactor for http logs
arukompas Jul 16, 2023
095f7ce
Merge branch 'access-logs' of github.com:opcodesio/log-viewer into ac…
arukompas Jul 16, 2023
1a40f4d
Fix styling
arukompas Jul 16, 2023
caed8ac
custom columns for the access/http-error logs
arukompas Jul 16, 2023
d7e07d8
bug fixes + sort severities
arukompas Jul 16, 2023
7a382c5
bug fixes, improvements
arukompas Jul 17, 2023
278ffc4
Fix styling
arukompas Jul 17, 2023
cf6834d
refactor, bug fixes, improvements
arukompas Jul 19, 2023
a498815
Merge branch 'access-logs' of github.com:opcodesio/log-viewer into ac…
arukompas Jul 19, 2023
a790d66
Fix styling
arukompas Jul 19, 2023
ce2f677
refactor, improvements to the log api
arukompas Jul 20, 2023
beaa885
Fix styling
arukompas Jul 20, 2023
215b5a4
caching improvements
arukompas Jul 21, 2023
cc17aac
Merge branch 'main' into access-logs
arukompas Jul 22, 2023
6152121
Fix styling
arukompas Jul 22, 2023
ab3a70e
add Horizon log support
arukompas Jul 22, 2023
d5bcd28
change successful log icon
arukompas Jul 23, 2023
e01ffab
add Postgres and PHP FPM log support
arukompas Jul 23, 2023
06ff6f9
support for Supervisor and Redis logs
arukompas Jul 23, 2023
e23ba21
Fix styling
arukompas Jul 23, 2023
e08ebbd
file type selector + bug fixes
arukompas Jul 23, 2023
d00f833
Merge branch 'access-logs' of github.com:opcodesio/log-viewer into ac…
arukompas Jul 23, 2023
390b6a8
Fix styling
arukompas Jul 23, 2023
e8458b2
fix log levels, add gzip compression on caches
arukompas Jul 28, 2023
83c653a
Merge branch 'access-logs' of github.com:opcodesio/log-viewer into ac…
arukompas Jul 28, 2023
df0fb1e
Fix styling
arukompas Jul 28, 2023
c927a6c
fix tests
arukompas Jul 28, 2023
154b914
Fix styling
arukompas Jul 28, 2023
be17f84
fix broken tests on Windows
arukompas Jul 28, 2023
e12d113
Merge branch 'access-logs' of github.com:opcodesio/log-viewer into ac…
arukompas Jul 28, 2023
0841880
fix tests for Windows
arukompas Jul 28, 2023
66ced08
debug test on windows
arukompas Jul 28, 2023
1abf37c
more debugging of windows test
arukompas Jul 28, 2023
aad97db
potential fix
arukompas Jul 28, 2023
b0258be
remove dumps
arukompas Jul 28, 2023
4af528c
potential fix
arukompas Jul 28, 2023
82f5b51
potential fix
arukompas Jul 28, 2023
a352d1d
remove unnecessary comment
arukompas Jul 29, 2023
d5e397f
Merge branch 'main' into access-logs
arukompas Jul 30, 2023
52bb16a
remove version from composer.json
arukompas Jul 30, 2023
afe670f
refactoring
arukompas Jul 30, 2023
8483006
refactor
arukompas Jul 30, 2023
31dc17d
bug fixes
arukompas Jul 30, 2023
ee41ead
Fix styling
arukompas Jul 30, 2023
8294890
update Pest
arukompas Jul 30, 2023
209e58b
Merge branch 'access-logs' of github.com:opcodesio/log-viewer into ac…
arukompas Jul 30, 2023
157830d
some small improvements, refactor
arukompas Aug 1, 2023
41287e6
update the GH action to include Laravel 10 and the different Pest ver…
arukompas Aug 1, 2023
f45dff7
also add nunomaduro/collision to the matrix
arukompas Aug 1, 2023
9a1a4d3
exclude testing Laravel 10 on PHP 8.0
arukompas Aug 1, 2023
7bfdf2d
fix test
arukompas Aug 1, 2023
b65dc76
second try
arukompas Aug 1, 2023
8bc3fcf
Merge branch 'main' into access-logs
arukompas Aug 11, 2023
1734d39
bug fixes after merge
arukompas Aug 11, 2023
44ad2ce
some typing improvements
arukompas Aug 11, 2023
fbc7d62
small changes
arukompas Aug 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/fix-php-code-style-issues.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
ref: ${{ github.head_ref }}

- name: Fix PHP code style issues
uses: aglipanci/laravel-pint-action@0.1.0
uses: aglipanci/laravel-pint-action@1.0.0

- name: Commit changes
uses: stefanzweifel/git-auto-commit-action@v4
Expand Down
13 changes: 11 additions & 2 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,20 @@ jobs:
matrix:
os: [ubuntu-latest, windows-latest]
php: [8.1, 8.0]
laravel: [9.*]
laravel: [9.*, 10.*]
stability: [prefer-lowest, prefer-stable]
exclude:
- php: 8.0
laravel: 10.*
include:
- laravel: 9.*
testbench: 7.*
pest: 1.*
collision: 6.*
- laravel: 10.*
testbench: 8.*
pest: 2.*
collision: 7.*

name: P${{ matrix.php }} - L${{ matrix.laravel }} - ${{ matrix.stability }} - ${{ matrix.os }}

Expand All @@ -40,7 +49,7 @@ jobs:

- name: Install dependencies
run: |
composer require "laravel/framework:${{ matrix.laravel }}" "orchestra/testbench:${{ matrix.testbench }}" --no-interaction --no-update
composer require "laravel/framework:${{ matrix.laravel }}" "orchestra/testbench:${{ matrix.testbench }}" "pestphp/pest:${{ matrix.pest }}" "pestphp/pest-plugin-laravel:${{ matrix.pest }}" "nunomaduro/collision:${{ matrix.collision }}" --no-interaction --no-update
composer update --${{ matrix.stability }} --prefer-dist --no-interaction

- name: Execute tests
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ phpstan.neon
testbench.yaml
vendor
node_modules
tests/Feature/performance_test.log
8 changes: 3 additions & 5 deletions composer.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
{
"name": "opcodesio/log-viewer",
"version": "v2.5.4",
"description": "Fast and easy-to-use log viewer for your Laravel application",
"keywords": [
"arukompas",
Expand All @@ -27,11 +26,10 @@
"guzzlehttp/guzzle": "^7.2",
"itsgoingd/clockwork": "^5.1",
"laravel/pint": "^1.0",
"nunomaduro/collision": "^6.0",
"nunomaduro/collision": "^7.0",
"orchestra/testbench": "^7.6|^8.0",
"pestphp/pest": "^1.21",
"pestphp/pest-plugin-laravel": "^1.1",
"phpunit/phpunit": "^9.5",
"pestphp/pest": "^2.0",
"pestphp/pest-plugin-laravel": "^2.0",
"spatie/test-time": "^1.3"
},
"suggest": {
Expand Down
44 changes: 13 additions & 31 deletions config/log-viewer.php
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
<?php

use Opcodes\LogViewer\Level;

return [

/*
Expand Down Expand Up @@ -134,6 +132,19 @@
'include_files' => [
'*.log',
'**/*.log',

// You can include paths to other log types as well, such as apache, nginx, and more.
'/var/log/httpd/*',
'/var/log/nginx/*',

// MacOS Apple Silicon logs
'/opt/homebrew/var/log/nginx/*',
'/opt/homebrew/var/log/httpd/*',
'/opt/homebrew/var/log/php-fpm.log',
'/opt/homebrew/var/log/postgres*log',
'/opt/homebrew/var/log/redis*log',
'/opt/homebrew/var/log/supervisor*log',

// '/absolute/paths/supported',
],

Expand Down Expand Up @@ -164,35 +175,6 @@
'/vendor/barryvdh/laravel-debugbar/',
],

/*
|--------------------------------------------------------------------------
| Log matching patterns
|--------------------------------------------------------------------------
| Regexes for matching log files
|
*/

'patterns' => [
'laravel' => [
'log_matching_regex' => '/^\[(\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}\.?(\d{6}([\+-]\d\d:\d\d)?)?)\].*/',

/**
* This pattern, used for processing Laravel logs, returns these results:
* $matches[0] - the full log line being tested.
* $matches[1] - full timestamp between the square brackets (includes microseconds and timezone offset)
* $matches[2] - timestamp microseconds, if available
* $matches[3] - timestamp timezone offset, if available
* $matches[4] - contents between timestamp and the severity level
* $matches[5] - environment (local, production, etc)
* $matches[6] - log severity (info, debug, error, etc)
* $matches[7] - the log text, the rest of the text.
*/
'log_parsing_regex' => '/^\[(\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}\.?(\d{6}([\+-]\d\d:\d\d)?)?)\](.*?(\w+)\.|.*?)('
.implode('|', array_filter(Level::caseValues()))
.')?: (.*?)( in [\/].*?:[0-9]+)?$/is',
],
],

/*
|--------------------------------------------------------------------------
| Cache driver
Expand Down
61 changes: 24 additions & 37 deletions phpunit.xml.dist
Original file line number Diff line number Diff line change
@@ -1,39 +1,26 @@
<?xml version="1.0" encoding="UTF-8"?>
<phpunit
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
backupGlobals="false"
backupStaticAttributes="false"
bootstrap="vendor/autoload.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="false"
executionOrder="random"
failOnWarning="true"
failOnRisky="true"
failOnEmptyTestSuite="true"
beStrictAboutOutputDuringTests="true"
verbose="true"
>
<testsuites>
<testsuite name="Opcodes LogViewer Test Suite">
<directory>tests</directory>
</testsuite>
</testsuites>
<coverage>
<include>
<directory suffix=".php">./src</directory>
</include>
<report>
<html outputDirectory="build/coverage"/>
<text outputFile="build/coverage.txt"/>
<clover outputFile="build/logs/clover.xml"/>
</report>
</coverage>
<logging>
<junit outputFile="build/report.junit.xml"/>
</logging>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.2/phpunit.xsd" backupGlobals="false" bootstrap="vendor/autoload.php" colors="true" processIsolation="false" stopOnFailure="false" executionOrder="random" failOnWarning="true" failOnRisky="true" failOnEmptyTestSuite="true" beStrictAboutOutputDuringTests="true" cacheDirectory=".phpunit.cache" backupStaticProperties="false">
<testsuites>
<testsuite name="Opcodes Log Viewer Test Suite">
<directory>tests</directory>
</testsuite>
</testsuites>
<coverage>
<report>
<html outputDirectory="build/coverage"/>
<text outputFile="build/coverage.txt"/>
<clover outputFile="build/logs/clover.xml"/>
</report>
</coverage>
<logging>
<junit outputFile="build/report.junit.xml"/>
</logging>
<php>
<env name="LOG_VIEWER_CACHE_DRIVER" value="array"/>
</php>
<source>
<include>
<directory suffix=".php">./src</directory>
</include>
</source>
</phpunit>
14 changes: 14 additions & 0 deletions pint.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"preset": "laravel",
"rules": {
"class_attributes_separation": {
"elements": {
"const": "none",
"method": "one",
"property": "none",
"trait_import": "none",
"case": "none"
}
}
}
}
2 changes: 1 addition & 1 deletion public/app.css

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion public/app.js

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions public/mix-manifest.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"/app.js": "/app.js?id=65bcb5cc7b0f371b0d28d1b07e92cc53",
"/app.css": "/app.css?id=93151d8b186ef7758df8582425ff8082",
"/app.js": "/app.js?id=e7bdf4c9df5545d0907880d43b7adf71",
"/app.css": "/app.css?id=2644d93568c08a41a0612c7c2c38618e",
"/img/log-viewer-128.png": "/img/log-viewer-128.png?id=d576c6d2e16074d3f064e60fe4f35166",
"/img/log-viewer-32.png": "/img/log-viewer-32.png?id=f8ec67d10f996aa8baf00df3b61eea6d",
"/img/log-viewer-64.png": "/img/log-viewer-64.png?id=8902d596fc883ca9eb8105bb683568c6"
Expand Down
2 changes: 1 addition & 1 deletion resources/css/app.scss
Original file line number Diff line number Diff line change
Expand Up @@ -323,7 +323,7 @@ html.dark {

.log-list {
table > thead th {
@apply sticky top-0 z-10 bg-gray-100 dark:bg-gray-900 py-2 px-2 text-left text-sm font-semibold text-gray-500 dark:text-gray-400;
@apply sticky top-0 z-10 bg-gray-100 dark:bg-gray-900 py-2 px-1 lg:px-2 text-left text-xs lg:text-sm font-semibold text-gray-500 dark:text-gray-400;
}

.log-group {
Expand Down
169 changes: 169 additions & 0 deletions resources/js/components/BaseLogTable.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
<template>
<table class="table-fixed min-w-full max-w-full border-separate" style="border-spacing: 0">
<thead class="bg-gray-50">
<tr>
<th class="hidden lg:table-cell"><span class="sr-only">Expand/Collapse</span></th>
<th v-for="(column) in logViewerStore.columns" scope="col">
<div>{{ column.label }}</div>
</th>
<th scope="col" class="hidden lg:table-cell"><span class="sr-only">Log index</span></th>
</tr>
</thead>

<template v-if="logViewerStore.logs && logViewerStore.logs.length > 0">
<tbody v-for="(log, index) in logViewerStore.logs" :key="index"
:class="[index === 0 ? 'first' : '', 'log-group']"
:id="`tbody-${index}`" :data-index="index"
>
<tr @click="logViewerStore.toggle(index)"
:class="['log-item group', log.level_class, logViewerStore.isOpen(index) ? 'active' : '', logViewerStore.shouldBeSticky(index) ? 'sticky z-2' : '']"
:style="{ top: logViewerStore.stackTops[index] || 0 }"
>
<td class="log-level hidden lg:table-cell">
<div class="flex items-center lg:pl-2">
<button :aria-expanded="logViewerStore.isOpen(index)"
@keydown="handleLogToggleKeyboardNavigation"
class="log-level-icon opacity-75 w-5 h-5 hidden lg:block group focus:opacity-100 focus:outline-none focus:ring-2 focus:ring-brand-500 rounded-md"
>
<span class="sr-only" v-if="!logViewerStore.isOpen(index)">Expand log entry</span>
<span class="sr-only" v-if="logViewerStore.isOpen(index)">Collapse log entry</span>
<span class="w-full h-full group-hover:hidden group-focus:hidden">
<ExclamationCircleIcon v-if="log.level_class === 'danger'" />
<ExclamationTriangleIcon v-else-if="log.level_class === 'warning'" />
<CheckCircleIcon v-else-if="log.level_class === 'success'" />
<InformationCircleIcon v-else />
</span>
<span class="w-full h-full hidden group-hover:inline-block group-focus:inline-block">
<ChevronRightIcon :class="[logViewerStore.isOpen(index) ? 'rotate-90' : '', 'transition duration-100']" />
</span>
</button>
</div>
</td>

<template v-for="(column, colIndex) in logViewerStore.columns">
<!-- Severity -->
<td :key="`${log.index}-column-${colIndex}`" v-if="column.data_path === 'level'" class="log-level truncate">
<span>{{ log.level_name }}</span>
</td>
<!-- /Severity -->

<!-- Datetime -->
<td :key="`${log.index}-column-${colIndex}`" v-else-if="column.data_path === 'datetime'" class="whitespace-nowrap text-gray-900 dark:text-gray-200">
<span class="hidden lg:inline" v-html="highlightSearchResult(log.datetime, searchStore.query)"></span>
<span class="lg:hidden">{{ log.time }}</span>
</td>
<!-- /Datetime -->

<!-- Message -->
<td :key="`${log.index}-column-${colIndex}`" v-else-if="column.data_path === 'message'" class="max-w-[1px] w-full truncate text-gray-500 dark:text-gray-300 dark:opacity-90">
<span v-html="highlightSearchResult(`${log.message}`, searchStore.query)"></span>
</td>
<!-- /Message -->

<td :key="`${log.index}-column-${colIndex}`" v-else class="text-gray-500 dark:text-gray-300 dark:opacity-90" :class="column.class || ''">
<span v-html="highlightSearchResult(getDataAtPath(log, column.data_path), searchStore.query)"></span>
</td>
</template>

<td class="whitespace-nowrap text-gray-500 dark:text-gray-300 dark:opacity-90 text-xs hidden lg:table-cell">
<LogCopyButton :log="log" class="pr-2 large-screen" />
</td>
</tr>
<tr v-show="logViewerStore.isOpen(index)">
<td colspan="6">
<div class="lg:hidden flex justify-between px-2 pt-2 pb-1 text-xs">
<div class="flex-1"><span class="font-semibold">Datetime:</span> {{ log.datetime }}</div>
<div>
<LogCopyButton :log="log" />
</div>
</div>
<pre class="log-stack" v-html="highlightSearchResult(log.full_text, searchStore.query)"></pre>
<template v-if="hasContext(log)">
<p class="mx-2 lg:mx-8 pt-2 border-t font-semibold text-gray-700 dark:text-gray-400 text-xs lg:text-sm">Context:</p>
<pre class="log-stack" v-html="highlightSearchResult(prepareContextForOutput(log.context), searchStore.query)"></pre>
</template>

<div v-if="log.extra && log.extra.log_text_incomplete" class="py-4 px-8 text-gray-500 italic">
The contents of this log have been cut short to the first {{ LogViewer.max_log_size_formatted }}.
The full size of this log entry is <strong>{{ log.extra.log_size_formatted }}</strong>
</div>
</td>
</tr>
</tbody>
</template>

<tbody v-else class="log-group">
<tr>
<td colspan="6">
<div class="bg-white text-gray-600 dark:bg-gray-800 dark:text-gray-200 p-12">
<div class="text-center font-semibold">No results</div>
<div class="text-center mt-6">
<button v-if="searchStore.query?.length > 0"
class="px-3 py-2 border dark:border-gray-700 text-gray-800 dark:text-gray-200 hover:border-brand-600 dark:hover:border-brand-700 rounded-md"
@click="clearQuery">Clear search query
</button>
<button v-if="searchStore.query?.length > 0 && fileStore.selectedFile"
class="px-3 ml-3 py-2 border dark:border-gray-700 text-gray-800 dark:text-gray-200 hover:border-brand-600 dark:hover:border-brand-700 rounded-md"
@click.prevent="clearSelectedFile">Search all files
</button>
<button
v-if="severityStore.levelsFound.length > 0 && severityStore.levelsSelected.length === 0"
class="px-3 ml-3 py-2 border dark:border-gray-700 text-gray-800 dark:text-gray-200 hover:border-brand-600 dark:hover:border-brand-700 rounded-md"
@click="severityStore.selectAllLevels">Select all severities
</button>
</div>
</div>
</td>
</tr>
</tbody>
</table>
</template>

<script setup>
import {
ChevronRightIcon,
ExclamationCircleIcon,
ExclamationTriangleIcon,
CheckCircleIcon,
InformationCircleIcon,
} from '@heroicons/vue/24/solid';
import { highlightSearchResult } from '../helpers.js';
import { useLogViewerStore } from '../stores/logViewer.js';
import { useSearchStore } from '../stores/search.js';
import { useFileStore } from '../stores/files.js';
import LogCopyButton from './LogCopyButton.vue';
import { handleLogToggleKeyboardNavigation } from '../keyboardNavigation';
import { useSeverityStore } from '../stores/severity.js';

const fileStore = useFileStore();
const logViewerStore = useLogViewerStore();
const searchStore = useSearchStore();
const severityStore = useSeverityStore();
const emit = defineEmits(['clearSelectedFile', 'clearQuery']);

const clearSelectedFile = () => {
emit('clearSelectedFile');
}

const clearQuery = () => {
emit('clearQuery');
}

const getDataAtPath = (obj, path) => {
return String(path.split('.').reduce((acc, part) => acc && acc[part], obj));
}

const hasContext = (log) => {
return log.context && Object.keys(log.context).length > 0;
}

const prepareContextForOutput = (context) => {
return JSON.stringify(context, function (key, value) {
if (typeof value === 'string') {
return value.replaceAll('\n', '<br/>');
}

return value;
}, 2);
}
</script>
Loading
Loading