Oracle Cloud Infrastructure Logging Data Source

Introduction

Grafana is a popular technology that makes it easy to visualize logs and metrics. The OCI Logging Grafana Plugin can be used to extend Grafana by adding Oracle Cloud Infrastructure Logging as a data source in Grafana.

The plugin allows you to retrieve logs related to a number of resources on Oracle Cloud: Compute, Networking, Storage, custom logs from your application, and audit logs generated by Oracle Cloud services. Once these logs are in Grafana, they can be analyzed along with metrics, giving you a single pane of glass for your application monitoring needs.

Latest plugin version 4.X.X (available on Grafana Marketplace) is compatible with Grafana 10.

Breaking change! In case you are migrating from a previous version (2.x.x or below) of the OCI Logging Grafana Plugin and are not using Instance Principals (Authentication Provider not set as OCI instance), please refer to the Migration Instructions for Grafana OCI Logging Data Source Settings (User Principals and Single Tenancy mode only) because you will have to reconfigure the plugin setup.

For custom logs from your application, see Custom Logging on OCI.

Prerequisites

We will discuss two different Grafana IAM configurations that need to be in place, for Grafana to fetch the logs from Oracle Cloud Logging Service.

For local/dev box environment and Grafana Cloud

Installation

Please refers to the following compatibility matrix to choose plugin version accordingly to your Grafana installation: Compatibility Matrix

In order to simplify the installation process, we created detailed guides for you to follow.

Depending on your installation scenario you can use the following guides:

Install Grafana and the OCI Logs plugin on a Linux host using this document.
Install Grafana and the OCI Logs plugin on Grafana Cloud using this document.
Install Grafana and the OCI Logs plugin on a MacOS host using this document.

Configure OCI Identity Policies

In the OCI console under Identity > Groups click Create Group and create a new group called GrafanaLoggingUserGroup. Add the user configured in the OCI CLI to the newly-created group.

Under the Policy tab switch to the root compartment and click Create Policy. Create a policy allowing the group to read tenancy log groups and log content. Add the following policy statements:

allow group GrafanaLoggingUserGroup to read log-groups in tenancy
allow group GrafanaLoggingUserGroup to read log-content in tenancy
allow group GrafanaLoggingUserGroup to read compartments in tenancy

For compute-instance/VM on Oracle Cloud Infrastructure

Create a Dynamic Group for your instance

Provision an Oracle Linux virtual machine in OCI connected to a Virtual Cloud Network with access to the public internet. If you do not already have access to a Virtual Cloud Network with access to the public internet you can navigate to Virtual Cloud Networks under Networking and click Create Virtual Cloud Network. Choosing the CREATE VIRTUAL CLOUD NETWORK PLUS RELATED RESOURCES option will result in a VCN with an Internet Routing Gateway and Route Tables configured for access to the public internet. Three subnets will be created: one in each availability domain in the region.

After creating your VM, the next step is to create a dynamic group used to group virtual machine or bare metal compute instances as “principals” (similar to user groups).

You can define the dynamic group similarly to below, where your instance is part of the compartment given in the definition of the dynamic group.

Create an IAM policy for Dynamic Group for your instance

Next, create a policy named “grafana_policy” in the root compartment of your tenancy to permit instances in the dynamic group to make API calls against Oracle Cloud Infrastructure services. Add the following policy statements:

allow dynamicgroup DynamicGroupForGrafanaInstances to read log-groups in tenancy
allow dynamicgroup DynamicGroupForGrafanaInstances to read log-content in tenancy
allow dynamicgroup DynamicGroupForGrafanaInstances to read compartments in tenancy