Getting Started with Logs

Docs > Getting Started > Getting Started with Logs

Overview

Use Datadog Log Management, also called logs, to collect logs across multiple logging sources, such as your server, container, cloud environment, application, or existing log processors and forwarders. With conventional logging, you have to choose which logs to analyze and retain to maintain cost-efficiency. With Datadog Logging without Limits*, you can collect, process, archive, explore, and monitor your logs without logging limits.

This page shows you how to get started with Log Management in Datadog. If you haven’t already, create a Datadog account.

Configure a logging source

With Log Management, you can analyze and explore data in the Log Explorer, connect Tracing and Metrics to correlate valuable data across Datadog, and use ingested logs for Datadog Cloud SIEM. The lifecycle of a log within Datadog begins at ingestion from a logging source.

Server

There are several integrations available to forward logs from your server to Datadog. Integrations use a log configuration block in their conf.yaml file, which is available in the conf.d/ folder at the root of your Agent’s configuration directory, to forward logs to Datadog from your server.

logs:
  - type: file
    path: /path/to/your/integration/access.log
    source: integration_name
    service: integration_name
    sourcecategory: http_web_access

To begin collecting logs from a server:

If you haven’t already, install the Datadog Agent based on your platform.
Note: Log collection requires Datadog Agent v6+.
Collecting logs is not enabled by default in the Datadog Agent. To enable log collection, set logs_enabled to true in your datadog.yaml file.
Restart the Datadog Agent.
Follow the integration activation steps or the custom files log collection steps on the Datadog site.
Note: If you’re collecting logs from custom files and need examples for tail files, TCP/UDP, journald, or Windows Events, see Custom log collection.

Container

As of Datadog Agent v6, the Agent can collect logs from containers. Each containerization service has specific configuration instructions based where the Agent is deployed or run, or how logs are routed.

For example, Docker has two different types of Agent installation available: on your host, where the Agent is external to the Docker environment, or deploying a containerized version of the Agent in your Docker environment.

Kubernetes requires that the Datadog Agent run in your Kubernetes cluster, and log collection can be configured using a DaemonSet spec, Helm chart, or with the Datadog Operator.

To begin collecting logs from a container service, follow the in-app instructions.

Cloud

You can forward logs from multiple cloud providers, such as AWS, Azure, and Google Cloud, to Datadog. Each cloud provider has its own set of configuration instructions.

For example, AWS service logs are usually stored in S3 buckets or CloudWatch Log groups. You can subscribe to these logs and forward them to an Amazon Kinesis stream to then forward them to one or multiple destinations. Datadog is one of the default destinations for Amazon Kinesis Delivery streams.

To begin collecting logs from a cloud service, follow the in-app instructions.

Client

Datadog permits log collection from clients through SDKs or libraries. For example, use the datadog-logs SDK to send logs to Datadog from JavaScript clients.

To begin collecting logs from a client, follow the in-app instructions.

Other

If you’re using existing logging services or utilities such as rsyslog, Fluentd, or Logstash, Datadog offers plugins and log forwarding options.

If you don’t see your integration, you can type it in the other integrations box and get notifications for when the integration is available.

To begin collecting logs from a cloud service, follow the in-app instructions.

Explore your logs

Once a logging source is configured, your logs are available in the Log Explorer. This is where you can filter, aggregate, and visualize your logs.

For example, if you have logs flowing in from a service that you wish to examine further, filter by service. You can further filter by status, such as ERROR, and select Group into Patterns to see which part of your service is logging the most errors.

Filtering in the Log Explorer by error pattern

Aggregate your logs into Fields and visualize as Top List to see your top logging services. Select a source, such as info or warn, and select View Logs from the dropdown menu. The side panel populates logs based on error, so you quickly see which host and services require attention.

The Log Explorer offers the following features for log troubleshooting and exploration:

Advanced search and filtering with facets and queries
Log Analytics for grouping logs into patterns and aggregating data
Visualizations to display log data in various formats
Saved Views to save and share your search configurations
Export options to reuse your queries in different contexts

For detailed information about all Log Explorer features, see the Log Explorer documentation.

What’s next?

Once a logging source is configured, and your logs are available in the Log Explorer, you can begin to explore a few other areas of log management.

Log configuration

Set attributes and aliasing to unify your logs environment.
Control how your logs are processed with pipelines and processors.
As Logging without Limits* decouples log ingestion and indexing, you can configure your logs and choose which logs to index, retain, or archive.

Log correlation

Connect logs and traces to exact logs associated with a specific env, service, or version.
If you’re already using metrics in Datadog, you can correlate logs and metrics to gain context of an issue.