-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
91044e2
commit 6a40445
Showing
2 changed files
with
169 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,157 @@ | ||
--- | ||
sidebar_position: 2 | ||
title: Explore Best Practices | ||
description: Best practices in Log management and Explore | ||
image: https://dytvr9ot2sszz.cloudfront.net/logz-docs/social-assets/docs-social.jpg | ||
keywords: [logz.io, explore, dashboard, log analysis, observability] | ||
--- | ||
|
||
Once you've sent your data to Logz.io, you can search and query your logs to identify, debug, and monitor issues as quickly and effectively as possible. | ||
|
||
Explore supports a few query methods, including: | ||
|
||
|
||
## Simple | ||
|
||
Logz.io offers an intuative and easy way to build your query. Click the search bar or start typing to find and select fields, conditions, and values. | ||
|
||
Build your query by selecting fields, parameters, and conditions. To add a value that doesn't appear in your logs, type its name and click on the + sign. You can also add free text to your search, which will convert it into a Lucene query. | ||
|
||
/// ADD MORE INFO HERE | ||
|
||
|
||
## Lucene | ||
|
||
Logz.io supports Lucene, an open-source search engine software library. | ||
|
||
With Lucene, you can search for free text by typing the text string you want to find; for example, `error` will return all words containing this string, and using quotation marks, `"error"`, will return only the specific word you're searching for. | ||
|
||
![See error](https://dytvr9ot2sszz.cloudfront.net/logz-docs/explore-dashboard/best-error-aug27.png) | ||
|
||
Use the filters to refine your search. For example, you can filter out all of the eventType fields that are `Modified`. | ||
|
||
![Filter out](https://dytvr9ot2sszz.cloudfront.net/logz-docs/explore-dashboard/isnot-filter-aug27.png) | ||
|
||
The filters include numeric based fields, such as `LogSize`. Choose the operator and value to view the relevant results. | ||
|
||
![numeric filters](https://dytvr9ot2sszz.cloudfront.net/logz-docs/explore-dashboard/logsize-explore-aug27.png) | ||
|
||
|
||
|
||
|
||
|
||
### Apply regex to search | ||
|
||
:::caution | ||
Using Regex can overload your system and cause performance issues in your account. If Regex is necessary, it's best to apply filters and use shorter timeframes. | ||
::: | ||
|
||
Logz.io uses Apache Lucene's regular expression engine to parse regex queries, supporting regexp and query_string. | ||
|
||
While Lucene's regex supports all Unicode characters, several characters are reserved as operators and cannot be searched on their own: | ||
|
||
`. ? + * | { } [ ] ( ) " \` | ||
|
||
Depending on the optional operators enabled, some additional characters may also be reserved. These characters are: | ||
|
||
`# @ & < > ~` | ||
|
||
However, you can still use reserved characters by applying a backslash or double-quotes. For example: | ||
|
||
`\*` will render as a * sign. | ||
|
||
`\#` will render as a # sign. | ||
|
||
`\()` will render as brackets. | ||
|
||
|
||
To use Regex in a search query in OpenSearch, you'll need to use the following template: | ||
|
||
`fieldName:/.*value.*/`. | ||
|
||
For example, you have a field called `sentence` that holds the following line: "The quick brown fox jumps over the lazy dog". | ||
|
||
To find one of the values in the field, such as `fox`, you'll need to use the following query: | ||
|
||
`sentence:/.*fox.*/`. | ||
|
||
// DIDNT EDIT BELOW THIS LINE | ||
|
||
## Enrich log results | ||
|
||
You can add additional columns to your logs field view. | ||
|
||
Find the field you'd like to add, hover over it and click on the **+** button. | ||
|
||
![Add field](https://dytvr9ot2sszz.cloudfront.net/logz-docs/kibana-discover/add-field-discover.png) | ||
|
||
Once the field is added, you can move or remove it using its inner menu. | ||
|
||
![Edit field](https://dytvr9ot2sszz.cloudfront.net/logz-docs/kibana-discover/add-field-overview.gif) | ||
|
||
Finally, you can save your search and its view by clicking on the **Save** option, at the top navigation bar. | ||
|
||
![Save field](https://dytvr9ot2sszz.cloudfront.net/logz-docs/kibana-discover/save-your-fields.png) | ||
|
||
## Filter log results | ||
|
||
To narrow down your search, click the **Add filter** option underneath the search bar. | ||
|
||
Choose the field, operator, and value you'd like to apply in your filter, and click save. You can also create a custom label to rename the filter for better identification. | ||
|
||
![Apply a filter](https://dytvr9ot2sszz.cloudfront.net/logz-docs/kibana-discover/add-a-filter.png) | ||
|
||
Once you've set your filter, clicking on it will open additional abilities such as pinning it across all apps, excluding results, temporarily disabling it, editing, or deleting it. | ||
|
||
## Select logs' time frame | ||
|
||
The default period to display results is 15 minutes. You can edit this time frame by clicking on the **Show dates** link or clicking on the calendar icon. | ||
|
||
The calendar icon offers popular time frames for you to choose from and lets you select the refresh rate of your data. | ||
|
||
![Time frame options](https://dytvr9ot2sszz.cloudfront.net/logz-docs/kibana-discover/quick-time-edits.png) | ||
|
||
The **Show dates** option lets you set a start and end time. In the popup, select between the following options: | ||
|
||
* **Relative** - Set a start and end date to view your data | ||
* **Now** - Get real-time troubleshooting and monitoring of your logs | ||
* **Absolute** - Browse the calendar view and choose any time frame to view your data. In this option, you can type the time frame you want to view | ||
|
||
![Choose time frame](https://dytvr9ot2sszz.cloudfront.net/logz-docs/kibana-discover/time-settings-gif.gif) | ||
|
||
## Create Log Visualizations | ||
|
||
In the following video, you'll be able to see how to create a visualization dashboard based on your logs: | ||
|
||
|
||
<div> | ||
<video width="100%" height="auto" controls autoplay loop muted> | ||
<source src="https://dytvr9ot2sszz.cloudfront.net/logz-docs/videos/log-visualizations-velcfd5tpr.mp4" type="video/mp4" /> | ||
Your browser does not support the video tag. | ||
</video> | ||
</div> | ||
|
||
|
||
|
||
<div style={{position: 'relative', paddingBottom: '56.25%'}}> | ||
<iframe style={{position: 'absolute', top: '0', left: '0', width: '100%', height: '100%'}} src="https://fast.wistia.com/embed/iframe/velcfd5tpr" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> | ||
</div> | ||
|
||
## Divide your log data | ||
|
||
You can divide logs from different environments by type, by utilizing Logz.io's sub accounts option. | ||
|
||
Create a sub account and configure it to receive the same logs as an existing account, mapping it as a different data type. | ||
|
||
For example, if a `metadata` field is assigned as an `Object` in your production environment, you can assign it as a `String` in your testing environment by creating a sub account to which you’ll send the same logs. | ||
|
||
You can also send data from each environment to a dedicated sub account to monitor them individually. | ||
|
||
Learn more about [creating and managing sub accounts](/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts) and about [field mapping](/docs/user-guide/data-hub/field-mapping/) in your account. | ||
|
||
## Additional resources | ||
|
||
* [Configure an alert](https://docs.logz.io/docs/user-guide/log-management/log-alerts/configure-alert/) | ||
* [Use Insights to detect new exceptions and critical errors](https://docs.logz.io/docs/user-guide/log-management/insights/ai-insights/) | ||
* [Use Live tail to get a live view of your logs](https://docs.logz.io/docs/user-guide/log-management/live-tail/) | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -34,26 +34,30 @@ If you prefer to send your data manually, Logz.io offers numerous methods to do | |
| --- | --- | --- | --- | | ||
|[Filebeat](https://app.logz.io/#/dashboard/integrations/Filebeat-data)|[.NET](https://app.logz.io/#/dashboard/integrations/dotnet)|[Jaeger installation](https://app.logz.io/#/dashboard/integrations/Jaeger-data)|[Cloudflare](https://app.logz.io/#/dashboard/integrations/Cloudflare-network) | ||
|[S3 Bucket](https://app.logz.io/#/dashboard/integrations/AWS-S3-Bucket)|[Prometheus](https://app.logz.io/#/dashboard/integrations/Prometheus-remote-write)|[OpenTelemetry installation](https://app.logz.io/#/dashboard/integrations/OpenTelemetry-data)|[NGINX](https://app.logz.io/#/dashboard/integrations/Nginx-load) | ||
|[cURL](https://app.logz.io/#/dashboard/integrations/cURL-data)|[Azure Kubernetes Service](https://app.logz.io/#/dashboard/integrations/Kubernetes)|[Docker](https://app.logz.io/#/dashboard/integrations/Docker)|[Active directory](https://app.logz.io/#/dashboard/integrations/Active-Directory) | ||
|[JSON uploads](https://app.logz.io/#/dashboard/integrations/JSON)|[Google Kubernetes Engine over OpenTelemetry](https://app.logz.io/#/dashboard/integrations/Kubernetes)|[Kubernetes](https://app.logz.io/#/dashboard/integrations/Kubernetes)|[CloudTrail](https://app.logz.io/#/dashboard/integrations/AWS-CloudTrail) | ||
|[Docker container](https://app.logz.io/#/dashboard/integrations/Docker)|[Amazon EC2](https://app.logz.io/#/dashboard/integrations/AWS-EC2)|[Go instrumentation](https://app.logz.io/#/dashboard/integrations/GO)|[Auditbeat](https://app.logz.io/#/dashboard/integrations/auditbeat) | | ||
|[cURL](https://app.logz.io/#/dashboard/integrations/cURL-data)|[Java](https://app.logz.io/#/dashboard/integrations/Java)|[Docker](https://app.logz.io/#/dashboard/integrations/Docker)|[Active directory](https://app.logz.io/#/dashboard/integrations/Active-Directory) | ||
|[HTTP uploads](https://app.logz.io/#/dashboard/integrations/HTTP)|[Node.js](https://app.logz.io/#/dashboard/integrations/Node-js)|[Kubernetes](https://app.logz.io/#/dashboard/integrations/Kubernetes)|[CloudTrail](https://app.logz.io/#/dashboard/integrations/AWS-CloudTrail) | ||
|[Python](https://app.logz.io/#/dashboard/integrations/Python)|[Amazon EC2](https://app.logz.io/#/dashboard/integrations/AWS-EC2)|[Go instrumentation](https://app.logz.io/#/dashboard/integrations/GO)|[Auditbeat](https://app.logz.io/#/dashboard/integrations/auditbeat) | | ||
|
||
Browse the complete list of available shipping methods [here](https://docs.logz.io/docs/category/send-your-data/). | ||
Browse the complete list of available shipping methods [here](https://app.logz.io/#/dashboard/integrations/collectors). | ||
|
||
To learn more about shipping your data, check out **Shipping Log Data to Logz.io**: | ||
<!-- To learn more about shipping your data, check out **Shipping Log Data to Logz.io**: | ||
<div style={{position: 'relative', paddingBottom: '56.25%'}}> | ||
<iframe style={{position: 'absolute', top: '0', left: '0', width: '100%', height: '100%'}} src="https://fast.wistia.com/embed/iframe/oi6qydmyk6" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> | ||
</div> | ||
--> | ||
|
||
### Parsing your data | ||
|
||
Logz.io offers automatic parsing [for over 50 log types](https://docs.logz.io/docs/user-guide/data-hub/log-parsing/default-parsing/). | ||
|
||
If you can't find your log type, or if you're interested in sending custom logs, Logz.io will parse the logs for you. Parsing-as-a-service is included in your Logz.io subscription; just open a chat with our **Support team** with your request, you can also email us at [[email protected]](mailto:[email protected]). | ||
|
||
###### Additional resources | ||
<h4 id="logs-resources"> Additional resources </h4> | ||
|
||
//START HERE | ||
|
||
|
||
Learn more about sending data to Logz.io: | ||
|
@@ -65,9 +69,9 @@ Learn more about sending data to Logz.io: | |
|
||
### Explore your data with Logz.io's Log Management platform | ||
|
||
Logz.io’s **[Log Management](https://app.logz.io/#/dashboard/osd)** is where you can search and query log files. You can use it to identify and analyze your code, and the platform is optimized for debugging and troubleshooting issues as quickly and effectively as possible. | ||
Logz.io's [Explore](https://app.logz.io/#/dashboard/explore) is where you can view, search, and query your data. Use it to identify and analyze your code, debug and troubleshoot issues, and get recommendations to next steps with its AI Assistant. | ||
|
||
![Log management overview](https://dytvr9ot2sszz.cloudfront.net/logz-docs/accounts/log-analytics-main-sep26.png) | ||
![Log management overview](https://dytvr9ot2sszz.cloudfront.net/logz-docs/accounts/explore-aug27.png) | ||
|
||
The following list contains some of the common abilities available in Log Management: | ||
|
||
|