Write shell script to analysis log file
1
The log file is as below:-
Source=Mobile
IP=189.23.45.01
STATUS=SUCCESS
TIME=10 sec
Source=Desktop
IP=189.23.34.23
STATUS=FAIL
TIME=101 sec
Source=Mobile
IP=189.23.34.23
STATUS=FAIL
TIME=29 sec
File keep going so on.
Questions:
- Find IP where status is FAIL?
- Find Ave time taken by all request where status is "success"?
- List how many logins were via Mobile and how much time did it took ?
shell-script
asked Jan 5 at 17:37
MachineMachine
62
add a comment |
1
The log file is as below:-
Source=Mobile
IP=189.23.45.01
STATUS=SUCCESS
TIME=10 sec
Source=Desktop
IP=189.23.34.23
STATUS=FAIL
TIME=101 sec
Source=Mobile
IP=189.23.34.23
STATUS=FAIL
TIME=29 sec
File keep going so on.
Questions:
- Find IP where status is FAIL?
- Find Ave time taken by all request where status is "success"?
- List how many logins were via Mobile and how much time did it took ?
shell-script
asked Jan 5 at 17:37
MachineMachine
62
Thats just a typo....
– Machine
Jan 5 at 17:42
If the entries of the log are consistent (four lines, third being status), then you can list only the failed ones by utilisinggreps context line controls. e.g.grep -B2 -A1 STATUS=FAIL
– steeling
Jan 5 at 17:51
@Machine Could you please clarify what you mean with "...and how much time did it took"? Do you want to know the overall time involved in Mobile-logins, successful and failed? An average? A overal or average calculation for only successful or only failed mobile logins?
– ozzy
Jan 5 at 18:16
add a comment |
1
1
1
The log file is as below:-
Source=Mobile
IP=189.23.45.01
STATUS=SUCCESS
TIME=10 sec
Source=Desktop
IP=189.23.34.23
STATUS=FAIL
TIME=101 sec
Source=Mobile
IP=189.23.34.23
STATUS=FAIL
TIME=29 sec
File keep going so on.
Questions:
- Find IP where status is FAIL?
- Find Ave time taken by all request where status is "success"?
- List how many logins were via Mobile and how much time did it took ?
shell-script
asked Jan 5 at 17:37
MachineMachine
62
The log file is as below:-
Source=Mobile
IP=189.23.45.01
STATUS=SUCCESS
TIME=10 sec
Source=Desktop
IP=189.23.34.23
STATUS=FAIL
TIME=101 sec
Source=Mobile
IP=189.23.34.23
STATUS=FAIL
TIME=29 sec
File keep going so on.
Questions:
- Find IP where status is FAIL?
- Find Ave time taken by all request where status is "success"?
- List how many logins were via Mobile and how much time did it took ?
shell-script
shell-script
asked Jan 5 at 17:37
MachineMachine
62
asked Jan 5 at 17:37
MachineMachine
62
edited Jan 5 at 17:57
Machine
asked Jan 5 at 17:37
MachineMachine
62
asked Jan 5 at 17:37
MachineMachine
62
asked Jan 5 at 17:37
MachineMachine
62
62
Thats just a typo....
– Machine
Jan 5 at 17:42
If the entries of the log are consistent (four lines, third being status), then you can list only the failed ones by utilisinggreps context line controls. e.g.grep -B2 -A1 STATUS=FAIL
– steeling
Jan 5 at 17:51
@Machine Could you please clarify what you mean with "...and how much time did it took"? Do you want to know the overall time involved in Mobile-logins, successful and failed? An average? A overal or average calculation for only successful or only failed mobile logins?
– ozzy
Jan 5 at 18:16
add a comment |
Thats just a typo....
– Machine
Jan 5 at 17:42
If the entries of the log are consistent (four lines, third being status), then you can list only the failed ones by utilisinggreps context line controls. e.g.grep -B2 -A1 STATUS=FAIL
– steeling
Jan 5 at 17:51
@Machine Could you please clarify what you mean with "...and how much time did it took"? Do you want to know the overall time involved in Mobile-logins, successful and failed? An average? A overal or average calculation for only successful or only failed mobile logins?
– ozzy
Jan 5 at 18:16
Thats just a typo....
– Machine
Jan 5 at 17:42
Thats just a typo....
– Machine
Jan 5 at 17:42
If the entries of the log are consistent (four lines, third being status), then you can list only the failed ones by utilising
greps context line controls. e.g. grep -B2 -A1 STATUS=FAIL– steeling
Jan 5 at 17:51
If the entries of the log are consistent (four lines, third being status), then you can list only the failed ones by utilising
greps context line controls. e.g. grep -B2 -A1 STATUS=FAIL– steeling
Jan 5 at 17:51
@Machine Could you please clarify what you mean with "...and how much time did it took"? Do you want to know the overall time involved in Mobile-logins, successful and failed? An average? A overal or average calculation for only successful or only failed mobile logins?
– ozzy
Jan 5 at 18:16
@Machine Could you please clarify what you mean with "...and how much time did it took"? Do you want to know the overall time involved in Mobile-logins, successful and failed? An average? A overal or average calculation for only successful or only failed mobile logins?
– ozzy
Jan 5 at 18:16
add a comment |
2 Answers
2
active
oldest
votes
1
You're really asking 3 questions - I'll get you started with the first one, and you should make an effort to solve the other two yourself using the same basic structure (there are plenty of examples on this site for using Awk to do numerical processing such as averaging):
Use Awk in paragraph mode (by unsetting the record separator, RS), splitting the record into fields using = and n (newline):
$ awk -vRS= -F'[=n]' '/STATUS=FAIL/{print $4}' file.log
189.23.34.23
189.23.34.23
answered Jan 5 at 18:01
steeldriversteeldriver
34.9k35184
@Steekdriver, i need your help to understanding what u meant by unsetting the record separator, RS.
– Machine
Jan 8 at 13:31
@Machine see for example 4.9 Multiple-Line Records in the GNU Awk User's Guide
– steeldriver
Jan 8 at 13:42
add a comment |
1
In an unpolished version, a Bash-script could look like this, assuming that your data is contained in datafile:
#!/bin/bash
printf "IPs where status is fail:n"
grep -z -oP 'IP=K.*n(?=STATUS=FAIL)' datafile
printf "Avg time taken by all requests where status is 'success':n"
grep -z -oP 'STATUS=SUCCESSnTIME=Kd+' datafile |
awk '{ total += $1; count++ } END { print ( count == 0 ? "NaN" : total/count); }'
printf "Number of logins (successful and failed) via Mobile:n"
grep -c 'Source=Mobile' datafile
A brief elucidation:
- Q2) Calculation of the Average time: the
grepcommand extracts the time values (which are assumed to be all in seconds). These values are piped into theawkcommand, which calculates their average, and then prints that average.
answered Jan 5 at 17:55
ozzyozzy
4855
@Machine What did you mean with "...and how time did they sent" ?
– ozzy
Jan 5 at 17:56
can u explain what exactly u doing in second command to find Avg?
– Machine
Jan 5 at 17:59
@Machine Of course, if you elucidate your question :-)
– ozzy
Jan 5 at 18:01
@steeldriver Sorry. It seems I spoiled the fun... Perhaps we should leave it to Machine to figure out what the commands do precisely. Specific questions will be answered then...
– ozzy
Jan 5 at 18:07
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f492687%2fwrite-shell-script-to-analysis-log-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
You're really asking 3 questions - I'll get you started with the first one, and you should make an effort to solve the other two yourself using the same basic structure (there are plenty of examples on this site for using Awk to do numerical processing such as averaging):
Use Awk in paragraph mode (by unsetting the record separator, RS), splitting the record into fields using = and n (newline):
$ awk -vRS= -F'[=n]' '/STATUS=FAIL/{print $4}' file.log
189.23.34.23
189.23.34.23
answered Jan 5 at 18:01
steeldriversteeldriver
34.9k35184
@Steekdriver, i need your help to understanding what u meant by unsetting the record separator, RS.
– Machine
Jan 8 at 13:31
@Machine see for example 4.9 Multiple-Line Records in the GNU Awk User's Guide
– steeldriver
Jan 8 at 13:42
add a comment |
1
You're really asking 3 questions - I'll get you started with the first one, and you should make an effort to solve the other two yourself using the same basic structure (there are plenty of examples on this site for using Awk to do numerical processing such as averaging):
Use Awk in paragraph mode (by unsetting the record separator, RS), splitting the record into fields using = and n (newline):
$ awk -vRS= -F'[=n]' '/STATUS=FAIL/{print $4}' file.log
189.23.34.23
189.23.34.23
answered Jan 5 at 18:01
steeldriversteeldriver
34.9k35184
@Steekdriver, i need your help to understanding what u meant by unsetting the record separator, RS.
– Machine
Jan 8 at 13:31
@Machine see for example 4.9 Multiple-Line Records in the GNU Awk User's Guide
– steeldriver
Jan 8 at 13:42
add a comment |
1
1
1
You're really asking 3 questions - I'll get you started with the first one, and you should make an effort to solve the other two yourself using the same basic structure (there are plenty of examples on this site for using Awk to do numerical processing such as averaging):
Use Awk in paragraph mode (by unsetting the record separator, RS), splitting the record into fields using = and n (newline):
$ awk -vRS= -F'[=n]' '/STATUS=FAIL/{print $4}' file.log
189.23.34.23
189.23.34.23
answered Jan 5 at 18:01
steeldriversteeldriver
34.9k35184
You're really asking 3 questions - I'll get you started with the first one, and you should make an effort to solve the other two yourself using the same basic structure (there are plenty of examples on this site for using Awk to do numerical processing such as averaging):
Use Awk in paragraph mode (by unsetting the record separator, RS), splitting the record into fields using = and n (newline):
$ awk -vRS= -F'[=n]' '/STATUS=FAIL/{print $4}' file.log
189.23.34.23
189.23.34.23
answered Jan 5 at 18:01
steeldriversteeldriver
34.9k35184
answered Jan 5 at 18:01
steeldriversteeldriver
34.9k35184
answered Jan 5 at 18:01
steeldriversteeldriver
34.9k35184
answered Jan 5 at 18:01
steeldriversteeldriver
34.9k35184
34.9k35184
@Steekdriver, i need your help to understanding what u meant by unsetting the record separator, RS.
– Machine
Jan 8 at 13:31
@Machine see for example 4.9 Multiple-Line Records in the GNU Awk User's Guide
– steeldriver
Jan 8 at 13:42
add a comment |
@Steekdriver, i need your help to understanding what u meant by unsetting the record separator, RS.
– Machine
Jan 8 at 13:31
@Machine see for example 4.9 Multiple-Line Records in the GNU Awk User's Guide
– steeldriver
Jan 8 at 13:42
@Steekdriver, i need your help to understanding what u meant by unsetting the record separator, RS.
– Machine
Jan 8 at 13:31
@Steekdriver, i need your help to understanding what u meant by unsetting the record separator, RS.
– Machine
Jan 8 at 13:31
@Machine see for example 4.9 Multiple-Line Records in the GNU Awk User's Guide
– steeldriver
Jan 8 at 13:42
@Machine see for example 4.9 Multiple-Line Records in the GNU Awk User's Guide
– steeldriver
Jan 8 at 13:42
add a comment |
1
In an unpolished version, a Bash-script could look like this, assuming that your data is contained in datafile:
#!/bin/bash
printf "IPs where status is fail:n"
grep -z -oP 'IP=K.*n(?=STATUS=FAIL)' datafile
printf "Avg time taken by all requests where status is 'success':n"
grep -z -oP 'STATUS=SUCCESSnTIME=Kd+' datafile |
awk '{ total += $1; count++ } END { print ( count == 0 ? "NaN" : total/count); }'
printf "Number of logins (successful and failed) via Mobile:n"
grep -c 'Source=Mobile' datafile
A brief elucidation:
- Q2) Calculation of the Average time: the
grepcommand extracts the time values (which are assumed to be all in seconds). These values are piped into theawkcommand, which calculates their average, and then prints that average.
answered Jan 5 at 17:55
ozzyozzy
4855
@Machine What did you mean with "...and how time did they sent" ?
– ozzy
Jan 5 at 17:56
can u explain what exactly u doing in second command to find Avg?
– Machine
Jan 5 at 17:59
@Machine Of course, if you elucidate your question :-)
– ozzy
Jan 5 at 18:01
@steeldriver Sorry. It seems I spoiled the fun... Perhaps we should leave it to Machine to figure out what the commands do precisely. Specific questions will be answered then...
– ozzy
Jan 5 at 18:07
add a comment |
1
In an unpolished version, a Bash-script could look like this, assuming that your data is contained in datafile:
#!/bin/bash
printf "IPs where status is fail:n"
grep -z -oP 'IP=K.*n(?=STATUS=FAIL)' datafile
printf "Avg time taken by all requests where status is 'success':n"
grep -z -oP 'STATUS=SUCCESSnTIME=Kd+' datafile |
awk '{ total += $1; count++ } END { print ( count == 0 ? "NaN" : total/count); }'
printf "Number of logins (successful and failed) via Mobile:n"
grep -c 'Source=Mobile' datafile
A brief elucidation:
- Q2) Calculation of the Average time: the
grepcommand extracts the time values (which are assumed to be all in seconds). These values are piped into theawkcommand, which calculates their average, and then prints that average.
answered Jan 5 at 17:55
ozzyozzy
4855
@Machine What did you mean with "...and how time did they sent" ?
– ozzy
Jan 5 at 17:56
can u explain what exactly u doing in second command to find Avg?
– Machine
Jan 5 at 17:59
@Machine Of course, if you elucidate your question :-)
– ozzy
Jan 5 at 18:01
@steeldriver Sorry. It seems I spoiled the fun... Perhaps we should leave it to Machine to figure out what the commands do precisely. Specific questions will be answered then...
– ozzy
Jan 5 at 18:07
add a comment |
1
1
1
In an unpolished version, a Bash-script could look like this, assuming that your data is contained in datafile:
#!/bin/bash
printf "IPs where status is fail:n"
grep -z -oP 'IP=K.*n(?=STATUS=FAIL)' datafile
printf "Avg time taken by all requests where status is 'success':n"
grep -z -oP 'STATUS=SUCCESSnTIME=Kd+' datafile |
awk '{ total += $1; count++ } END { print ( count == 0 ? "NaN" : total/count); }'
printf "Number of logins (successful and failed) via Mobile:n"
grep -c 'Source=Mobile' datafile
A brief elucidation:
- Q2) Calculation of the Average time: the
grepcommand extracts the time values (which are assumed to be all in seconds). These values are piped into theawkcommand, which calculates their average, and then prints that average.
answered Jan 5 at 17:55
ozzyozzy
4855
In an unpolished version, a Bash-script could look like this, assuming that your data is contained in datafile:
#!/bin/bash
printf "IPs where status is fail:n"
grep -z -oP 'IP=K.*n(?=STATUS=FAIL)' datafile
printf "Avg time taken by all requests where status is 'success':n"
grep -z -oP 'STATUS=SUCCESSnTIME=Kd+' datafile |
awk '{ total += $1; count++ } END { print ( count == 0 ? "NaN" : total/count); }'
printf "Number of logins (successful and failed) via Mobile:n"
grep -c 'Source=Mobile' datafile
A brief elucidation:
- Q2) Calculation of the Average time: the
grepcommand extracts the time values (which are assumed to be all in seconds). These values are piped into theawkcommand, which calculates their average, and then prints that average.
answered Jan 5 at 17:55
ozzyozzy
4855
edited Jan 5 at 18:05
answered Jan 5 at 17:55
ozzyozzy
4855
answered Jan 5 at 17:55
ozzyozzy
4855
answered Jan 5 at 17:55
ozzyozzy
4855
4855
@Machine What did you mean with "...and how time did they sent" ?
– ozzy
Jan 5 at 17:56
can u explain what exactly u doing in second command to find Avg?
– Machine
Jan 5 at 17:59
@Machine Of course, if you elucidate your question :-)
– ozzy
Jan 5 at 18:01
@steeldriver Sorry. It seems I spoiled the fun... Perhaps we should leave it to Machine to figure out what the commands do precisely. Specific questions will be answered then...
– ozzy
Jan 5 at 18:07
add a comment |
@Machine What did you mean with "...and how time did they sent" ?
– ozzy
Jan 5 at 17:56
can u explain what exactly u doing in second command to find Avg?
– Machine
Jan 5 at 17:59
@Machine Of course, if you elucidate your question :-)
– ozzy
Jan 5 at 18:01
@steeldriver Sorry. It seems I spoiled the fun... Perhaps we should leave it to Machine to figure out what the commands do precisely. Specific questions will be answered then...
– ozzy
Jan 5 at 18:07
@Machine What did you mean with "...and how time did they sent" ?
– ozzy
Jan 5 at 17:56
@Machine What did you mean with "...and how time did they sent" ?
– ozzy
Jan 5 at 17:56
can u explain what exactly u doing in second command to find Avg?
– Machine
Jan 5 at 17:59
can u explain what exactly u doing in second command to find Avg?
– Machine
Jan 5 at 17:59
@Machine Of course, if you elucidate your question :-)
– ozzy
Jan 5 at 18:01
@Machine Of course, if you elucidate your question :-)
– ozzy
Jan 5 at 18:01
@steeldriver Sorry. It seems I spoiled the fun... Perhaps we should leave it to Machine to figure out what the commands do precisely. Specific questions will be answered then...
– ozzy
Jan 5 at 18:07
@steeldriver Sorry. It seems I spoiled the fun... Perhaps we should leave it to Machine to figure out what the commands do precisely. Specific questions will be answered then...
– ozzy
Jan 5 at 18:07
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f492687%2fwrite-shell-script-to-analysis-log-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Thats just a typo....
– Machine
Jan 5 at 17:42
If the entries of the log are consistent (four lines, third being status), then you can list only the failed ones by utilising
greps context line controls. e.g.grep -B2 -A1 STATUS=FAIL– steeling
Jan 5 at 17:51
@Machine Could you please clarify what you mean with "...and how much time did it took"? Do you want to know the overall time involved in Mobile-logins, successful and failed? An average? A overal or average calculation for only successful or only failed mobile logins?
– ozzy
Jan 5 at 18:16