The Legato Logging API provides a toolkit allowing code to be instrumented with error, warning, informational, and debugging messages. These messages can be turned on or off remotely and pushed or pulled from the device through a secure shell, cloud services interfaces, e-mail, SMS, etc.

Logging Basics

Legato's logging can be configured through this API, and there's also a command-line target log tool available.

Levels

Log messages are categorized according to the severity of the information being logged. A log message may be purely informational, describing something that is expected to occur from time-to-time during normal operation; or it may be a report of a fault that might have a significant negative impact on the operation of the system. To differentiate these, each log entry is associated with one of the following log levels:

DEBUG: Handy for troubleshooting.
INFORMATION: Expected to happen; can be interesting even when not troubleshooting.
WARNING: Should not normally happen; may not have any real impact on system performance.
ERROR: Fault that may result in noticeable short-term system misbehaviour. Needs attention.
CRITICAL: Fault needs urgent attention. Will likely result in system failure.
EMERGENCY: Definite system failure.

Standard Out and Standard Error in Syslog

By default, app processes will have their stdout and stderr redirected to the syslog. Each process’s stdout will be logged at INFO severity level; it’s stderr will be logged at “ERR” severity level.

There are two limitations with this feature:

the PID reported in the logs generally refer to the PID of the process that generates the stdout/stderr message. If a process forks, then both the parent and child processes’ stdout/stderr will share the same connection to the syslog, and the parent’s PID will be reported in the logs for both processes.
stdout is line buffered when connected to a terminal, which means printf(“hello ”) will be printed to the terminal immediately. If stdout is connected to something like a pipe it's bulk buffered, which means a flush doesn't occur until the buffer is full.

To make your process line buffer stdout so that printf will show up in the logs as expected, the setlinebuf(stdout) system call can be used. Alternatively, fflush(stdout) can be called \ to force a flush of the stdout buffer.

This issue doesn't exist with stderr as stderr is never buffered.

Basic Logging

A series of macros are available to make logging easy.

None of them return anything.

All of them accept printf-style arguments, consisting of a format string followed by zero or more parameters to be printed (depending on the contents of the format string).

There is a logging macro for each of the log levels:

LE_DEBUG(formatString, ...)
LE_INFO(formatString, ...)
LE_WARN(formatString, ...)
LE_ERROR(formatString, ...)
LE_CRIT(formatString, ...)
LE_EMERG(formatString, ...)

For example,

LE_INFO("Obtained new IP address %s.", ipAddrStr);

Conditional Logging

Similar to the basic macros, but these contain a conditional expression as their first parameter. If this expression equals true, then the macro will generate this log output:

LE_DEBUG_IF(expression, formatString, ...)
LE_INFO_IF(expression, formatString, ...)
LE_WARN_IF(expression, formatString, ...)
LE_ERROR_IF(expression, formatString, ...)
LE_CRIT_IF(expression, formatString, ...)
LE_EMERG_IF(expression, formatString, ...)

Instead of writing

if (result == -1)
{
    LE_WARN("Failed to send message to server.  Errno = %m.");
}

you could write this:

LE_WARN_IF(result == -1, "Failed to send message to server. Errno = %m.");

Fatal Errors

There are some special logging macros intended for fatal errors:

LE_FATAL(formatString, ...)
Always kills the calling process after logging the message at EMERGENCY level (never returns).
LE_FATAL_IF(condition, formatString, ...)
If the condition is true, kills the calling process after logging the message at EMERGENCY level.
LE_ASSERT(condition)
If the condition is true, does nothing. If the condition is false, logs the source code text of the condition at EMERGENCY level and kills the calling process.
LE_ASSERT_OK(condition)
If the condition is LE_OK (0), does nothing. If the condition is anything else, logs the a message at EMERGENCY level, containing the source code text of the condition, indicating that it did not evaluate to LE_OK, and kills the calling process.

For example,

if (NULL == objPtr)
{
    LE_FATAL("Object pointer is NULL!");
}
// Now I can go ahead and use objPtr, knowing that if it was NULL then LE_FATAL() would have
// been called and LE_FATAL() never returns.

or,

LE_FATAL_IF(NULL == objPtr, "Object pointer is NULL!");
// Now I can go ahead and use objPtr, knowing that if it was NULL then LE_FATAL_IF() would not
// have returned.

or,

LE_ASSERT(NULL != objPtr);
// Now I can go ahead and use objPtr, knowing that if it was NULL then LE_ASSERT() would not
// have returned.

Tracing

Finally, a macro is provided for tracing:

LE_TRACE(traceRef, string, ...)

This macro is special because it's independent of log level. Instead, trace messages are associated with a trace keyword. Tracing can be enabled and disabled based on these keywords.

If a developer wanted to trace the creation of "shape" objects in their GUI package, they could add trace statements like the following:

LE_TRACE(NewShapeTraceRef, "Created %p with position (%d,%d).", shapePtr, shapePtr->x, shapePtr->y);

The reference to the trace is obtained at start-up as follows:

NewShapeTraceRef = le_log_GetTraceRef("newShape");

This allows enabling and disabling these LE_TRACE() calls using the "newShape" keyword through configuration settings and runtime log control tools. See Log Controls below.

Applications can use LE_IS_TRACE_ENABLED(NewShapeTraceRef) to query whether a trace keyword is enabled.

These allow apps to hook into the trace management system to use it to implement sophisticated, app-specific tracing or profiling features.

Result Code Text

The le_result_t macro supports printing an error condition in a human-readable text string.

result = le_foo_DoSomething();
if (result != LE_OK)
{
    LE_ERROR("Failed to do something. Result = %d (%s).", result, LE_RESULT_TXT(result));
}

Log Controls

Log level filtering and tracing can be controlled at runtime using:

the command-line Log Control Tool (log)
configuration settings
environment variables
function calls.

Log Control Tool

The log control tool is used from the command-line to control the log level filtering, log output location (syslog/stderr), and tracing for different components within a running system.

Online documentation can be accessed from the log control tool by running "log help".

Here are some code samples.

To set the log level to INFO for a component "myComp" running in all processes with the name "myProc":

$ log level INFO myProc/myComp

To set the log level to DEBUG for a component "myComp" running in a process with PID 1234:

$ log level DEBUG 1234/myComp

To enable all LE_TRACE statements tagged with the keyword "foo" in a component called "myComp" running in all processes called "myProc":

$ log trace foo myProc/myComp

To disable the trace statements tagged with "foo" in the component "myComp" in processes called "myProc":

$ log stoptrace foo myProc/myComp

With all of the above examples "*" can be used in place of the process name or a component name (or both) to mean "all processes" and/or "all components".

Log Control Configuration Settings

Note: The configuration settings haven't been implemented yet.

Environment Variables

Environment variables can be used to control the default log settings, taking effect immediately at process start-up; even before the Log Control Daemon has been connected to.

Settings in the Log Control Daemon (applied through configuration and/or the log control tool) will override the environment variable settings when the process connects to the Log Control Daemon.

LE_LOG_LEVEL

LE_LOG_LEVEL can be used to set the default log filter level for all components in the process. Valid values are:

EMERGENCY
CRITICAL
ERROR
WARNING
INFO
DEBUG

For example,

$ export LE_LOG_LEVEL=DEBUG

LE_LOG_TRACE

LE_LOG_TRACE allows trace keywords to be enabled by default. The contents of this variable is a colon-separated list of keywords that should be enabled. Each keyword must be prefixed with a component name followed by a slash ('/').

For example,

$ export LE_LOG_TRACE=framework/fdMonitor:framework/logControl

Programmatic Log Control

Normally, configuration settings and the log control tool should suffice for controlling logging functionality. In some situations, it can be convenient to control logging programmatically in C.

le_log_SetFilterLevel() sets the log filter level.

le_log_GetFilterLevel() gets the log filter level.

Trace keywords can be enabled and disabled programmatically by calling le_log_EnableTrace() and le_log_DisableTrace().

Log Formats

Log entries can also contain any of these:

timestamp (century, year, month, day, hours, minutes, seconds, milliseconds, microseconds)
level (debug, info, warning, etc.) or trace keyword
process ID
component name
thread name
source code file name
function name
source code line number

Log messages have the following format:

Jan  3 02:37:56  INFO  | processName[pid]/componentName T=threadName | fileName.c funcName() lineNum | Message

App Crash Logs

When a process within an app faults or exits in error, a copy of the current syslog buffer is captured along with a core file of the process crash (if generated).

The core file maximum size is determined by the process settings maxCoreDumpFileBytes and maxFileBytes found in the processes section of your app's .adef file. By default, the maxCoreDumpFileBytes is set to 0, do not create a core file.

To help save the target from flash burnout, the syslog and core files are stored in the RAM FS under /tmp. When a crash occurs, this directory is created:

 /tmp/legato_logs/<app-name>/<exe-name>/

The path for myApp and myExe would be:

 /tmp/legato_logs/myApp/myExe/

The files in that directory look like this:

 core-myExe-1418694851
 syslog-1418694851

If the fault action for that app's process is to reboot the target, the output location is changed to this (and is preserved across reboots):

 /mnt/flash/legato_logs/<app-name>/<exe-name>/

To save on RAM and flash space, only the most recent 4 copies of each file are preserved.

Copyright (C) Sierra Wireless Inc. Use of this work is subject to license.