parsing log4j log file using regular expression

Question

I have created a java application for parsing the log4j log file using regular expression, The application is working fine for the log which i have shown below

1999-11-27 15:49:37,459 [thread-x] ERROR mypackage - Catastrophic system failure

but not working for

2015-01-22 01:52:54,237 [http-bio-80-exec-5] FATAL   TestLog4jServlet - Show FATAL message

My log4j ConversionPattern is given below

log4j.appender.Appender2.layout.ConversionPattern=%d [%t] %-7p %10c{1} - %m%n

Can anyone please tell me some solution for this

My code is as given below

public static void main(String[] args) {
    String regex = "(\\d{4}-\\d{2}-\\d{2}) (\\d{2}:\\d{2}:\\d{2},\\d{3}) \\[(.*)\\] ([^ ]*) ([^ ]*) - (.*)$";

    Pattern p = Pattern.compile(regex);
    String[] samples = {
            "2015-01-22 01:52:54,237 [http-bio-80-exec-5] FATAL   TestLog4jServlet - Show FATAL message"
        };

    Matcher m = p.matcher(samples[1]);
    System.out.println(m.matches());
    if (m.matches() && m.groupCount() == 6) {
        String date = m.group(1);
        String time = m.group(2);
        String threadId = m.group(3);
        String priority = m.group(4);
        String category = m.group(5);
        String message = m.group(6);

        System.out.println("date: " + date);
        System.out.println("time: " + time);
        System.out.println("threadId: " + threadId);
        System.out.println("priority: " + priority);
        System.out.println("category: " + category);
        System.out.println("message: " + message);
    }
}

Avinash Raj · Accepted Answer · 2015-01-22 08:26:04Z

4

Because there are two spaces between FATAL and TestLog4jServlet but you included only one space in your regex. So i suggest you to replace the corresponding space with <space>+ which allows one or more spaces.

(\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2},\d{3}) \[(.*?)\] ([^ ]*) +([^ ]*) - (.*)$
                                                                ^
                                                                |

DEMO

Java regex would be,

"(\\d{4}-\\d{2}-\\d{2}) (\\d{2}:\\d{2}:\\d{2},\\d{3}) \\[(.*)\\] ([^ ]*) +([^ ]*) - (.*)$"

edited Jan 22, 2015 at 8:26

answered Jan 22, 2015 at 8:07

Avinash Raj

175k32 gold badges247 silver badges289 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Zheng HU · Accepted Answer · 2015-01-22 08:30:03Z

0

I think the Logstash is better for parsing log.

answered Jan 22, 2015 at 8:30

Zheng HU

53 bronze badges

2 Comments

Zheng HU Over a year ago

Create a conf file like logstash.conf with the content below :

input { 	file { 		path => "/your/log/absolupath" 	} }  filter { 	grok { 		match => [ "message","%{DATA:date} \[%{DATA:threadId}\]\s+%{DATA:priority}\s+%{DATA:category}\s+-\s+%{GREEDYDATA:message}"  ] 	} 	date {         match => [ "date", "yyyy-MM-dd HH:mm:ss,SSS" ]     } }  output { 	file { 		path => "/your/output/absolupath" 	} }

And then run the logstash : ./logstash agent -f logstash.conf, you can find in your output file a json data for the parsed log.

Will Over a year ago

@Alex I think Zheng is saying: "don't try to parse it yourself. Set up and use Logstash to parse your log files". You might want it if you are centralising your logs / making them searchable, not sure what your use case is. Note: it's not answering your question directly

Mahesh Revaskar · Accepted Answer · 2023-12-10 20:05:34Z

0

added group names to the detected regex patterns Regex for parsing log4j log

(?<date>\d{4}-\d{2}-\d{2}) +(?<time>\d{2}:\d{2}:\d{2},\d{3}) +(\[thread-(?<threadId>.*?)\])(?<logLevel>\S*) (?<package>(.*?)) - (?<msg>.*)

edited Dec 10, 2023 at 20:05

answered Oct 17, 2023 at 14:03

Mahesh Revaskar

1041 silver badge5 bronze badges

Collectives™ on Stack Overflow

parsing log4j log file using regular expression

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related