Linux command to find the number of open connections for apache

July 24, 2010 · Leave a Comment
Filed under: Apache, Featured 

To find the no of apache connections

For all Linux , Ubuntu , Red hat system we have check the number of connections in apache server using the below command.  This returns the count of apache threads.




IF you server don’t have httpd service then try for “apache” too

ps ax|grep httpd | wc -l

FIXED: Windows could not start the Subversion Apache on Local Computer

October 24, 2009 · Leave a Comment
Filed under: Apache, Featured, VB.Net 

FIXED: Windows could not start the Subversion Apache on Local Computer The Error: Windows could not start the CollabNet Subversion Apache on Local Computer. For more information, review the System Event Log. If this is a non-Microsoft service, contact the service vendor, and refer to service-specific error code 1.

How to Installing Apache 2.2.11 with Weblogic 10.3 on 64 bit Ubuntu 9.04

October 11, 2009 · 1 Comment
Filed under: Apache, Featured, Ubuntu, Weblogic, WebServers 

To install Apache 2.2.11 on Ubuntu 9.04 64 bit and make it work with Weblogic 10.3 please follow the below steps. The post walks through the steps that I followed, the problems that I faced and the solutions ( er..hacks) to get them resolved.

Installing Apache

  • Is simple do

sudo apt-get install apache2

If you want to build it from sources then follow these steps.

  • Once you have installed apache2 then the installation happens at the following locations in Ubuntu

Apache config files are in /etc/apache
Apache log files are in /var/log/apache
Apache libs are in /usr/lib/apache
Other files can be in /usr/share/apache, /var/lib/apache
executables in /usr/sbin apache and apache2ctl

  • Now to start apache execute the following

stev@stev-laptop:/usr/sbin$ sudo apache2ctl start

  • Note that the server is started as a root, else you might get the following error

(13)Permission denied: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs

You should be able to access the default page on http://localhost now and see It Works!

Now, Integrating with Weblogic


  • Download the apache plugins for weblogic 10.3 from the following location

http://download.oracle.com/otn/bea/weblogic/server103/server103_apacheplugins.zip

  • Unzip the contents and select the mod_wl_22.so file from the location

<apache-plugin-unzip-location>linux/x86_64

  • This file needs to be loaded the following location

/usr/lib/apache2/modules

  • to test whether the file is valid or not do ldd

stev@stev-laptop:/usr/lib/apache2/modules$ ldd mod_wl_22.so
linux-vdso.so.1 =>  (0×00007fff02ffe000)
libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0×00007f5cfaaff000)
libm.so.6 => /lib/libm.so.6 (0×00007f5cfa87a000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0×00007f5cfa661000)
libc.so.6 => /lib/libc.so.6 (0×00007f5cfa2ef000)
/lib64/ld-linux-x86-64.so.2 (0×00007f5cfafc5000)

  • You might notice that some file like in my case libstdc++.so.5 => was not found!

-bash-3.00$ ldd mod_wl_22.so
libstdc++.so.5 => not found
libm.so.6 => /lib64/tls/libm.so.6 (0×0000002a9572f000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0×0000002a958b5000)
libc.so.6 => /lib64/tls/libc.so.6 (0×0000002a959c3000)
/lib64/ld-linux-x86-64.so.2 (0×000000552aaaa000)

  • Goto the synaptic package manager, search and install this file.
  • Now create a weblogic.load file with the following contents

LoadModule weblogic_module /usr/lib/apache2/modules/mod_wl_22.so

  • and put it at the following location

/etc/apache2/mods-enabled

  • You can also create a weblogic.conf file with the following contents at the same location

<Location /medrec>
SetHandler weblogic-handler
WebLogicHost 127.0.1.1
WebLogicPort 7011
</Location>

This would help in redirecting a request like http://localhost/medrec to the weblogic server running on 127.0.1.1 and at 7011 port with the application name medrec.

  • Restart Apache

stev@stev-laptop:/usr/sbin$ sudo apache2ctl restart

  • Check whether the weblogic_module has been loaded successfully or not

stev@stev-laptop:/usr/sbin$ apache2 -t -D DUMP_MODULES
apache2: Could not reliably determine the server’s fully qualified domain name, using 127.0.1.1 for ServerName
Loaded Modules:
core_module (static)
log_config_module (static)
logio_module (static)
mpm_worker_module (static)
http_module (static)
so_module (static)
alias_module (shared)
auth_basic_module (shared)
authn_file_module (shared)
authz_default_module (shared)
authz_groupfile_module (shared)
authz_host_module (shared)
authz_user_module (shared)
autoindex_module (shared)
cgid_module (shared)
deflate_module (shared)
dir_module (shared)
env_module (shared)
mime_module (shared)
negotiation_module (shared)
setenvif_module (shared)
status_module (shared)
weblogic_module (shared)

  • If you see the weblogic_module loaded fine then you have configured it correctly and can now access the application at

http://localhost/medrec/

  • To test whether the apache2.conf file is correct or not you can use

stev@stev-laptop:/usr/sbin$ apache2 -t
apache2: Could not reliably determine the server’s fully qualified domain name, using 127.0.1.1 for ServerName
Syntax OK

Other Potential Problem(s)

  • You may encounter the problem bad user name ${APACHE_RUN_USER}

this is because the value of ${APACHE_RUN_USER} is not getting picked up. Ideally it should be picked up from /etc/apache2/envvars

which has the following entries

# envvars – default environment variables for apache2ctl

# Since there is no sane way to get the parsed apache2 config in scripts, some
# settings are defined via environment variables and then used in apache2ctl,
# /etc/init.d/apache2, /etc/logrotate.d/apache2, etc.
export APACHE_RUN_USER=www-data
export APACHE_RUN_GROUP=www-data
export APACHE_PID_FILE=/var/run/apache2.pid

However, still in your case if these values are not picked up then goto the /etc/apache2/apache2.conf file and make the following changes

# These need to be set in /etc/apache2/envvars
# User ${APACHE_RUN_USER}
# Group ${APACHE_RUN_GROUP}
User www-data
Group www-data

  • The weblogic.load file is not read for some reason and hence the weblogic_module is not loaded

Make an entry in the httpd.conf file at the following location

/etc/apache2/httpd.conf

and enter the LoadModule and other configuration data in this file

LoadModule weblogic_module /usr/lib/apache2/modules/mod_wl_22.so

<Location /medrec>
SetHandler weblogic-handler
WebLogicHost 127.0.1.1
WebLogicPort 7011
</Location>

There is an entry in the apache2.conf file to include the httpd.conf file, hence the changes of httpd.conf would get loaded

# Include all the user configurations:
Include /etc/apache2/httpd.conf

Author :  Vikas Hazrati

Vikas Hazrati

Apache Lucene Java Search Engine in Version 2.9 – Linux Magazine

September 29, 2009 · Leave a Comment
Filed under: Apache, Featured, JAVA 

The free Java search engine Apache Lucene is available in version 2.9. The release provides numerous new features and API enhancements.

Among the new features to Apache Lucene are per segment searching and more scalable multi-term queries. New analyzers are available for Persian, Arabic and Chinese. Also, developers increased the speed of numeric range searching so that they need not be converted to text and are indexed in a tree structure for faster processing.

Version 2.9 is not completely backward-compatible. Details are in the Lucene changelog. The team recommends compiling applications with Lucene 2.9 and not simply swap out the binaries. That way problems with backward compatibility can be ironed out more easily.

The next Lucene release should bear the number 3.0, require Java 1.5 and remove “deprecated” code.

Lucene 2.9 is under Apache License 2.0 and available at a number of mirrors for download.

Apaches beat Red Devils for first time in history

September 5, 2009 · Leave a Comment
Filed under: Apache, Featured 

ATKINS — The Pottsville Apaches beat the Red Devils 14-6 for the first time in history before a packed Lemley Field crowd Friday night.
“This does not start tonight at 7 p.m. This started back in January, and you have worked and worked and worked. Now is the fun part,” Pottsville (1-0) coach Bryan Rust told his team before the game’s opening kickoff.
The evidence of the Apaches hard work was evident as they ran 74 offensive plays compared to 51 plays for the Red Devils (0-1). Pottsville overpowered Atkins and used long drives to keep the Red Devils’ speed at bay to grind out a victory.
The first quarter looked like a heavyweight fight as both teams were able to drive the ball, but neither could capitalize on field position or opponent’s mistakes. The Apaches scored on their first possession of the second quarter when Tyler Robertson broke through the pile on the left side for a 23-yard touchdown run with 8:54 left in the half. He also barreled up the middle for the conversion, giving Pottsville their first lead of the game at 8-0.
Atkins was not going to be outdone and roared back when D.J. Glover scored on a second and 8 yards to go from the 28. The 2-point conversion attempt was stopped a foot short. The score remained 8-0 until halftime.
Pottsville began to flex its muscles in the third quarter and kept the ball away from the Red Devils by grinding out 23 plays and holding Atkins to just eight plays on offense. Apache Corey Taylor capped an 8-play drive that covered 59 yards. Taylor and Robertson combined for all the yardage on the drive, while Taylor finished it on a 16-yard scamper around the right side. That touchdown proved to be the final blow as the Pottsville ran the ball — and the clock — the rest of the second half.
Atkins’ last attempt at a drive was stopped after 67 yards when John Campbell intercepted a Red Devil pass at the Pottsville 21.
The Apaches dominated the ground game by piling up 292 yards rushing. Robertson led the way with 148 yards. Quarterback Derek Clark added 59, Jay Wright had 41 and Corey Taylor and Brandon Long chipped in with 10 and 34.
“It was a great game,” Rust said after the game. “It’s a big win for these kids, the school and the community. I am so proud of how these guys have worked. We have a chance to be pretty good, if we will continue to work hard.”
The Apaches have their home opener at 7 p.m. next week against Lamar at John Needham Stadium. Atkins will host the Dover Pirates in the annual Rivals Cup game at Lemley Field.

Configuring Apache2.2 mod_proxy_ajp with Tomcat

June 4, 2009 · Leave a Comment
Filed under: Featured, JSP, Linux, MySql, MySQL 5.1 

mod_proxy_ajp is an Apache module which can be used to forward a client HTTP request to an internal Tomcat application server using the AJP protocol.  In this example I have used JSPWki as the example application running on a Windows server with Apache2.2 and Tomcat 5.5

Advantages of mod_proxy_ajp rather:

  • You can gain a lot of flexibility (lot of the apache modules/features can be used especially “name-based virtual hosting”)
  • Practical for those who need to support Java applications along with PHP / Perl … (only one apache server is needed)
  • Certificates management is easier in apache configuration (this argument is a lot subjective)
  • It’s not Tomcat’s main objective to serve http static resources (not optimized for that)
  • Load balancing/cluster management is easier with an apache frontend

Tomcat configuration

We just have to create the AJP connector in the conf/server.xml file like that:

<Connector port="8009" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" />

This line will enable AJP connections to the 8009 port of your tomcat server (localhost for example).

Apache2 configuration

One way (useful if this apache is a global front end) is to create a virtual host for this application.

  • ProxyPass and ProxyPassReverse are classic reverse proxy directives used to forward the stream to another location.
  • ajp://… is the AJP connector location (your tomcat’s server host/port)

# JSP Wiki DNS

<VirtualHost 10.1.1.170:80>

ServerName wiki.example.com

ServerAlias wiki

DocumentRoot “D:/Tomcat 5.5/webapps/JSPWiki/”

#DocumentRoot “D:/Apache2.2/htdocs/JSPWiki”

<Proxy *>

AddDefaultCharset Off

Order deny,allow

Allow from all

</Proxy>

ProxyPass / ajp://localhost:8009/

ProxyPassReverse / ajp://localhost:8009/

</VirtualHost>

The following entries were made in server.xml on the Tomcat App Server :

<Host name=”wiki.example.com” debug=”0″

appBase=”D:/Tomcat 5.5/webapps”

unpackWARs=”true” autoDeploy=”true”>

<Alias>wiki</Alias>

<Context path= “” docBase=”D:/Tomcat 5.5/webapps/JSPWiki” debug=”1″/>

<Valve className=”org.apache.catalina.valves.AccessLogValve”

directory=”logs” prefix=”home_access_log.” suffix=”.txt”

pattern=”common” resolveHosts=”false”/>

</Host>

Under JSPWiki/WEB-INF/jspwiki.properties the Base URL was modified as follows :

jspwiki.baseURL=http://wiki/JSPWiki/

How to make Apache run ASP.NET / ASP.NET 2.0

December 15, 2008 · Leave a Comment
Filed under: Apache, ASP, ASP.NET, ASP.NET 2.0, Featured 

Even worked with ASP.NET 2.0 Site !

Following are the instruction to make Asp.Net work under apache:

– Install Apache 2.0.54

– Install Mod_AspDotNet

– Add at the end of C:Program FilesApache GroupApache2confhttpd.conf the following lines

#asp.net
LoadModule aspdotnet_module “modules/mod_aspdotnet.so”

AddHandler asp.net asax ascx ashx asmx aspx axd config cs csproj licx rem resources resx soap vb vbproj vsdisco webinfo

<IfModule mod_aspdotnet.cpp>
# Mount the ASP.NET /asp application
AspNetMount /SampleASP “c:/SampleASP”
#/SampleASP is the alias name for asp.net to execute
#”c:/SampleASP” is the actual execution of files/folders in that location

# Map all requests for /asp to the application files
Alias /SampleASP “c:/SampleASP”
#maps /SampleASP request to “c:/SampleASP”
#now to get to the /SampleASP type http://localhost/SampleASP
#It’ll redirect http://localhost/SampleASP to “c:/SampleASP”

# Allow asp.net scripts to be executed in the /SampleASP example
<Directory “c:/SampleASP”>
Options FollowSymlinks ExecCGI
Order allow,deny
Allow from all
DirectoryIndex index.htm index.aspx
#default the index page to .htm and .aspx
</Directory>

# For all virtual ASP.NET webs, we need the aspnet_client files
# to serve the client-side helper scripts.
AliasMatch /aspnet_client/system_web/(d+)_(d+)_(d+)_(d+)/(.*) “C:/Windows/Microsoft.NET/Framework/v$1.$2.$3/ASP.NETClientFiles/$4

<Directory “C:/Windows/Microsoft.NET/Framework/v*/ASP.NETClientFiles”>
Options FollowSymlinks
Order allow,deny
Allow from all
</Directory>
</IfModule>
#asp.net

– Create a directory c:SampleASP and insert in it the index.aspx

– Restart apache server :
Start-> Apache HTTP Server 2.0.54 ->
Control Apache Server -> Restart

– Open Explorer and navigate to http://localhost/SampleASP/index.aspx

If everything worked fine you should get a nice asp.net page working.

— index.aspx —

<%@ Page Language=”VB” %>
<html>
<head>
<link rel=”stylesheet”href=”intro.css”>
</head>
<body>
<center>
<form action=”index.aspx” method=”post”>
<h3> Name: <input id=”Name” type=text>
Category: <select id=”Category” size=1>
<option>One</option>
<option>Two</option>
<option>Three</option>
</select>
</h3>
<input type=submit value=”Lookup”>
<p>
<% Dim I As Integer
For I = 0 to 7 %>
<font size=”<%=I%>”> Sample ASP.NET TEST</font> <br>
<% Next %>
</form>
</center>
</body>
</html>

Apache FOP- PDF RTF generation for JAVA

December 15, 2008 · Leave a Comment
Filed under: Apache, Featured, FOP, JAVA, PDF 

Apache FOP

Domain path: for examples and packages:

\domainJAVACODE LIBRARYFOP_PDF_RTF_generationpackage

Introduction

Apache FOP (Formatting Objects Processor) is a print formatter driven by XSL formatting objects (XSL-FO) and an output independent formatter. It is a Java application that reads a formatting object (FO) tree and renders the resulting pages to a specified output. Output formats currently supported include PDF, PS, PCL, AFP, XML (area tree representation), Print, AWT and PNG, and to a lesser extent, RTF and TXT. The primary output target is PDF.

Render Diagram

A beta release of the latest version of FOP (0.95) is now available. After a short testing period it will become the third stable release after the large redesign effort and will implement a large subset of the XSL-FO Version 1.1 W3C Recommendation.

Support for each of the standard’s objects and properties is detailed in FOP Compliance. Download options include a precompiled version, source code, and many example files to get you started. Resources include links to XSL-FO introductions and many other useful references. A checklist for Getting Help will guide you toward maximizing the usefulness of FOP.

FOP is proud to be part of Apache’s XML Graphics project.

Demonstration

Formatting Diagram

This image is a demonstration of a real two page document. The xml data on the left is formatted into the two pages on the right. The document contains static areas that appear on every page, an external graphic, a footnote on the first page, and a table that goes across both pages.

FOP uses the standard XSL-FO file format as input, lays the content out into pages, then renders it to the requested output. One great advantage of using XSL-FO as input is that XSL-FO is itself an XML file, which means that it can be conveniently created from a variety of sources. The most common method is to convert semantic XML to XSL-FO, using an XSLT transformation.

FOP Objectives

The goals of the Apache FOP project are to deliver an XSL-FO to PDF formatter that is compliant to at least the Basic conformance level described in the W3C Recommendation from 05 December 2006, and that complies with the November 2001 Portable Document Format Specification (Version 1.4) from Adobe Systems.

Conformance to the XML 1.0 Recommendation, XSLT 1.0 Recommendation and the XML Namespaces Recommendation is understood. Other relevant documents, such as the XPath and XLink Working Drafts, are referenced as necessary. The FOP Project will attempt to use the latest version of evolving specifications.

Configure Web Logs in Apache

August 24, 2008 · Leave a Comment
Filed under: Apache, Featured, Linux 

One of the many pieces of the Website puzzle is Web logs. Traffic analysis is central to most Websites, and the key to getting the most out of your traffic analysis revolves around how you configure your Web logs.

Apache is one of the most — if not the most — powerful open source solutions for Website operations. You will find that Apache’s Web logging features are flexible for the single Website or for managing numerous domains requiring Web log analysis.

Author’s Note: While most of this piece discusses configuration options for any operating system Apache supports, some of the content will be Unix/Linux (*nix) specific, which now includes Macintosh OS X and its underlying Unix kernel.

For the single site, Apache is pretty much configured for logging in the default install. The initial httpd.conf file (found in /etc/httpd/conf/httpd.conf in most cases) should have a section on logs that looks similar to this (Apache 2.0.x), with descriptive comments for each item. Your default logs folder will be found in /etc/httpd/logs. This location can be changed when dealing with multiple Websites, as we’ll see later. For now, let’s review this section of log configuration.

ErrorLog logs/error_log

LogLevel warn

LogFormat “%h %l %u %t “%r” %>s %b “%{Referer}i” “%{User-Agent}i”” combined

LogFormat “%h %l %u %t “%r” %>s %b” common

LogFormat “%{Referer}i -> %U” referer

LogFormat “%{User-agent}i” agent

CustomLog logs/access_log combined

Error Logs

The error log contains messages sent from Apache for errors encountered during the course of operation. This log is very useful for troubleshooting Apache issues on the server side.

Apache Log Tip: If you are monitoring errors or testing your server, you can use the command line to interactively watch log entries. Open a shell session and type "tail –f /path/to/error_log". This will show you the last few entries in the file and also continue to show new entries as they occur.

There are no real customization options available, other than telling Apache where to establish the file, and what level of error logging you seek to capture. First, let’s look at the error log configuration code from httpd.conf.

ErrorLog logs/error_log

You may wish to store all error-related information in one error log. If so, the above is fine, even for multiple domains. However, you can specify an error log file for each individual domain you have. This is done in the <VirtualHost> container with an entry like this:

<VirtualHost 10.0.0.2>

DocumentRoot "/home/sites/domain1/html/"

ServerName domain1.com

ErrorLog /home/sites/domain1/logs/error.log

</VirtualHost>

If you are responsible for reviewing error log files as a server administrator, it is recommended that you maintain a single error log. If you’re hosting for clients, and they are responsible for monitoring the error logs, it’s more convenient to specify individual error logs they can access at their own convenience.

The setting that controls the level of error logging to capture follows below.

LogLevel warn

Apache’s definitions for their error log levels are as follows:

1299_apachelogstable1 (click to view image)

Tracking Website Activity

Often by default, Apache will generate three activity logs: access, agent and referrer. These track the accesses to your Website, the browsers being used to access the site and referring urls that your site visitors have arrived from.

It is commonplace now to utilize Apache’s “combined” log format, which compiles all three of these logs into one logfile. This is very convenient when using traffic analysis software as a majority of these third-party programs are easiest to configure and schedule when only dealing with one log file per domain.

Let’s break down the code in the combined log format and see what it all means.

LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" combined

LogFormat starts the line and simply tells Apache you are defining a log file type (or nickname), in this case, combined. Now let’s look at the cryptic symbols that make up this log file definition.

1299_apachelogstable2 (click to view image)

To review all of the available configuration codes for generating a custom log, see Apache’s docs on the module_log_config, which powers log files in Apache.

Apache Log Tip: You could capture more from the HTTP header if you so desired. A full listing and definition of data in the header is found at the World Wide Web Consortium.

For a single Website, the default entry would suffice:

CustomLog logs/access_log combined

However, for logging multiple sites, you have a few options. The most common is to identify individual log files for each domain. This is seen in the example below, again using the log directive within the <VirtualHost> container for each domain.

<VirtualHost 10.0.0.2>

DocumentRoot "/home/sites/domain1/html/"

ServerName domain1.com

ErrorLog /home/sites/domain1/logs/error.log

CustomLog /home/sites/domain1/logs/web.log

</VirtualHost>

<VirtualHost 10.0.0.3>

DocumentRoot “/home/sites/domain2/html/”

ServerName domain2.com

ErrorLog /home/sites/domain2/logs/error.log

CustomLog /home/sites/domain2/logs/web.log

</VirtualHost>

<VirtualHost 10.0.0.4>

DocumentRoot “/home/sites/domain3/html/”

ServerName domain3.com

ErrorLog /home/sites/domain3/logs/error.log

CustomLog /home/sites/domain3/logs/web.log

</VirtualHost>

In the above example, we have three domains with three unique Web logs (using the combined format we defined earlier). A traffic analysis package could then be scheduled to process these logs and generate reports for each domain independently.

This method works well for most hosts. However, there may be situations where this could become unmanageable. Apache recommends a special single log file for large virtual host environments and provides a tool for generating individual logs per individual domain.

We will call this log type the cvh for
mat, standing for “common virtual host.” Simply by adding a %v (which stands for virtual host) to the beginning of the combined log format defined earlier and giving it a new nickname of cvh, we can compile all domains into one log file, then automatically split them into individual log files for processing by a traffic analysis package.

LogFormat "%v %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" cvh

In this case, we do not make any CustomLog entries in the <VirtualHost> containers and simply have one log file generated by Apache. A program created by Apache called split_logfile is included in the src/support directory of your Apache sources. If you did not compile from source or do not have the sources, you can get the Perl script.

The individual log files created from your master log file will be named for each domain (virtual host) and look like: virtualhost.log.

Log Rotation

Finally, we want to address log rotation. High traffic sites will generate very large log files, which will quickly swallow up valuable disk space on your server. You can use log rotation to manage this process.

There are many ways to handle log rotation, and various third-party tools are available as well. However, we’re focusing on configurations native to Apache, so we will look at a simple log rotation scheme here. I’ll include links to more flexible and sophisticated log rotation options in a moment.

This example uses a rudimentary shell script to move the current Web log to an archive log, compresses the old file and keeps an archive for as long as 12 months, then restarts Apache with a pause to allow the log files to be switched out.

mv web11.tgz web12.tgz

mv web10.tgz web11.tgz

mv web9.tgz web10.tgz

mv web8.tgz web9.tgz

mv web7.tgz web8.tgz

mv web6.tgz web7.tgz

mv web5.tgz web6.tgz

mv web4.tgz web5.tgz

mv web3.tgz web4.tgz

mv web2.tgz web3.tgz

mv web1.tgz web2.tgz

mv web.tgz web1.tgz

mv web.log web.old

/usr/sbin/apachectl graceful

sleep 300

tar cvfz web.tgz web.old

This code can be copied into a file called logrotate.sh, and placed inside the folder where your web.log file is stored (or whatever you name your log file, e.g. access_log, etc.). Just be sure to modify for your log file names and also chmod (change permissions on the file) to 755 so it becomes an executable.

This works fine for a single busy site. If you have more complex requirements for log rotation, be sure to see some of the following sites. In addition, many Linux distributions now come with a log rotation included. For example, Red Hat 9 comes with logrotate.d, a log rotation daemon which is highly configurable. To find out more, on your Linux system with logrotate.d installed, type man logrotate.

Apache Module mod_log_config

August 24, 2008 · Leave a Comment
Filed under: Apache, Featured, Linux 

Apache Module mod_log_config

Summary

This module provides for flexible logging of client requests. Logs are written in a customizable format, and may be written directly to a file, or to an external program. Conditional logging is provided so that individual requests may be included or excluded from the logs based on characteristics of the request.

Three directives are provided by this module: TransferLog to create a log file, LogFormat to set a custom format, and CustomLog to define a log file and format in one step. The TransferLog and CustomLog directives can be used multiple times in each server to cause each request to be logged to multiple files.

Directives

Topics

See also

Custom Log Formats

The format argument to the LogFormat and CustomLog directives is a string. This string is used to log each request to the log file. It can contain literal characters copied into the log files and the C-style control characters “n” and “t” to represent new-lines and tabs. Literal quotes and back-slashes should be escaped with back-slashes.

The characteristics of the request itself are logged by placing “%” directives in the format string, which are replaced in the log file by the values as follows:

Format String Description
%% The percent sign (Apache 2.0.44 and later)
%...a Remote IP-address
%...A Local IP-address
%...B Size of response in bytes, excluding HTTP headers.
%...b Size of response in bytes, excluding HTTP headers. In CLF format, i.e. a ‘-‘ rather than a 0 when no bytes are sent.
%...{Foobar}C The contents of cookie Foobar in the request sent to the server.
%...D The time taken to serve the request, in microseconds.
%...{FOOBAR}e The contents of the environment variable FOOBAR
%...f Filename
%...h Remote host
%...H The request protocol
%...{Foobar}i The contents of Foobar: header line(s) in the request sent to the server.
%...l Remote logname (from identd, if supplied). This will return a dash unless IdentityCheck is set On.
%...m The request method
%...{Foobar}n The contents of note Foobar from another module.
%...{Foobar}o The contents of Foobar: header line(s) in the reply.
%...p The canonical port of the server serving the request
%...P The process ID of the child that serviced the request.
%...{format}P The process ID or thread id of the child that serviced the request. Valid formats are pid and tid. (Apache 2.0.46 and later)
%...q The query string (prepended with a ? if a query string exists, otherwise an empty string)
%...r First line of request
%...s Status. For requests that got internally redirected, this is the status of the *original* request — %...>s for the last.
%...t Time the request was received (standard english format)
%...{format}t The time, in the form given by format, which should be in strftime(3) format. (potentially localized)
%...T The time taken to serve the request, in seconds.
%...u Remote user (from auth; may be bogus if return status (%s) is 401)
%...U The URL path requested, not including any query string.
%...v The canonical ServerName of the server serving the request.
%...V The server name according to the UseCanonicalName setting.
%...X Connection status when response is completed:

X = connection aborted before the response completed.
+ = connection may be kept alive after the response is sent.
- = connection will be closed after the response is sent.

(This directive was %...c in late versions of Apache 1.3, but this conflicted with the historical ssl %...{var}c syntax.)

%...I Bytes received, including request and headers, cannot be zero. You need to enable mod_logio to use this.
%...O Bytes sent, including headers, cannot be zero. You need to enable mod_logio to use this.

The “” can be nothing at all (e.g., "%h %u %r %s %b"), or it can indicate conditions for inclusion of the item (which will cause it to be replaced with “-” if the condition is not met). The forms of condition are a list of HTTP status codes, which may or may not be preceded by “!”. Thus, “%400,501{User-agent}i” logs User-agent: on 400 errors and 501 errors (Bad Request, Not Implemented) only; “%!200,304,302{Referer}i” logs Referer: on all requests which did not return some sort of normal status.

The modifiers “<” and “>” can be used for requests that have been internally redirected to choose whether the original or final (respectively) request should be consulted. By default, the % directives %s, %U, %T, %D, and %r look at the original request while all others look at the final request. So for example, %>s can be used to record the final status of the request and %<u can be used to record the original authenticated user on a request that is internally redirected to an unauthenticated resource.

Note that in httpd 2.0 versions prior to 2.0.46, no escaping was performed on the strings from %...r, %...i and %...o. This was mainly to comply with the requirements of the Common Log Format. This implied that clients could insert control characters into the log, so you had to be quite careful when dealing with raw log files.

For security reasons, starting with 2.0.46, non-printable and other special characters are escaped mostly by using xhh sequences, where hh stands for the hexadecimal representation of the raw byte. Exceptions from this rule are " and which are escaped by prepending a backslash, and all whitespace characters which are written in their C-style notation (n, t etc).

Note that in httpd 2.0, unlike 1.3, the %b and %B format strings do not represent the number of bytes sent to the client, but simply the size in bytes of the HTTP response (which will differ, for instance, if the connection is aborted, or if SSL is used). The %O format provided by mod_logio will log the actual number of bytes sent over the network.

Some commonly used log format strings are:

Common Log Format (CLF)
"%h %l %u %t "%r" %>s %b"
Common Log Format with Virtual Host
"%v %h %l %u %t "%r" %>s %b"
NCSA extended/combined log format
"%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i""
Referer log format
"%{Referer}i -> %U"
Agent (Browser) log format
"%{User-agent}i"

Note that the canonical ServerName and Listen of the server serving the request are used for %v and %p respectively. This happens regardless of the UseCanonicalName setting because otherwise log analysis programs would have to duplicate the entire vhost matching algorithm in order to decide what host really served the request.

Security Considerations

See the security tips document for details on why your security could be compromised if the directory where logfiles are stored is writable by anyone other than the user that starts the server.

BufferedLogs Directive

Description: Buffer log entries in memory before writing to disk
Syntax: BufferedLogs On|Off
Default: BufferedLogs Off
Context: server config
Status: Base
Module: mod_log_config
Compatibility: Available in versions 2.0.41 and later.

The BufferedLogs directive causes mod_log_config to store several log entries in memory and write them toget
her to disk, rather than writing them after each request. On some systems, this may result in more efficient disk access and hence higher performance. It may be set only once for the entire server; it cannot be configured per virtual-host.

This directive is experimental and should be used with caution.

CookieLog Directive

Description: Sets filename for the logging of cookies
Syntax: CookieLog filename
Context: server config, virtual host
Status: Base
Module: mod_log_config
Compatibility: This directive is deprecated.

The CookieLog directive sets the filename for logging of cookies. The filename is relative to the ServerRoot. This directive is included only for compatibility with mod_cookies, and is deprecated.

CustomLog Directive

Description: Sets filename and format of log file
Syntax: CustomLog file|pipe format|nickname [env=[!]environment-variable]
Context: server config, virtual host
Status: Base
Module: mod_log_config

The CustomLog directive is used to log requests to the server. A log format is specified, and the logging can optionally be made conditional on request characteristics using environment variables.

The first argument, which specifies the location to which the logs will be written, can take one of the following two types of values:

file
A filename, relative to the ServerRoot.
pipe
The pipe character “|“, followed by the path to a program to receive the log information on its standard input.

Security:

If a program is used, then it will be run as the user who started httpd. This will be root if the server was started by root; be sure that the program is secure.

Note

When entering a file path on non-Unix platforms, care should be taken to make sure that only forward slashed are used even though the platform may allow the use of back slashes. In general it is a good idea to always use forward slashes throughout the configuration files.

The second argument specifies what will be written to the log file. It can specify either a nickname defined by a previous LogFormat directive, or it can be an explicit format string as described in the log formats section.

For example, the following two sets of directives have exactly the same effect:

# CustomLog with format nickname

LogFormat "%h %l %u %t "%r" %>s %b" common

CustomLog logs/access_log common

# CustomLog with explicit format string

CustomLog logs/access_log “%h %l %u %t “%r” %>s %b”

The third argument is optional and controls whether or not to log a particular request based on the presence or absence of a particular variable in the server environment. If the specified environment variable is set for the request (or is not set, in the case of a ‘env=!name‘ clause), then the request will be logged.

Environment variables can be set on a per-request basis using the mod_setenvif and/or mod_rewrite modules. For example, if you want to record requests for all GIF images on your server in a separate logfile but not in your main log, you can use:

SetEnvIf Request_URI .gif$ gif-image

CustomLog gif-requests.log common env=gif-image

CustomLog nongif-requests.log common env=!gif-image

Or, to reproduce the behavior of the old RefererIgnore directive, you might use the following:

SetEnvIf Referer example.com localreferer

CustomLog referer.log referer env=!localreferer

LogFormat Directive

Description: Describes a format for use in a log file
Syntax: LogFormat format|nickname [nickname]
Default: LogFormat "%h %l %u %t "%r" %>s %b"
Context: server config, virtual host
Status: Base
Module: mod_log_config

This directive specifies the format of the access log file.

The LogFormat directive can take one of two forms. In the fi
rst form, where only one argument is specified, this directive sets the log format which will be used by logs specified in subsequent TransferLog directives. The single argument can specify an explicit format as discussed in the custom log formats section above. Alternatively, it can use a nickname to refer to a log format defined in a previous LogFormat directive as described below.

The second form of the LogFormat directive associates an explicit format with a nickname. This nickname can then be used in subsequent LogFormat or CustomLog directives rather than repeating the entire format string. A LogFormat directive that defines a nickname does nothing else — that is, it only defines the nickname, it doesn’t actually apply the format and make it the default. Therefore, it will not affect subsequent TransferLog directives. In addition, LogFormat cannot use one nickname to define another nickname. Note that the nickname should not contain percent signs (%).

Example

LogFormat "%v %h %l %u %t "%r" %>s %b" vhost_common

TransferLog Directive

Description: Specify location of a log file
Syntax: TransferLog file|pipe
Context: server config, virtual host
Status: Base
Module: mod_log_config

This directive has exactly the same arguments and effect as the CustomLog directive, with the exception that it does not allow the log format to be specified explicitly or for conditional logging of requests. Instead, the log format is determined by the most recently specified LogFormat directive which does not define a nickname. Common Log Format is used if no other format has been specified.

Example

LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i""

TransferLog logs/access_log

Next Page »