Download to read offline
![Chapter 1 ■ Background
10
This chart was created in response to the Fukushima Daiichi nuclear disaster of 2011, and sought to clear up
misinformation and misunderstanding of comparisons being made around the disaster. It did this by demonstrating the
differences in scale for the amount of radiation from sources such as other people or a banana, up to what a fatal dose of
radiation ultimately would be—how all that compared to spending just ten minutes near the Chernobyl meltdown.
Over the last quarter of a century, Edward Tufte, author and professor emeritus at Yale University, has been
working to raise the bar of information graphics. He published groundbreaking books detailing the history of data
visualization, tracing its roots even further back than Playfair, to the beginnings of cartography. Among his principles
is the idea to maximize the amount of information included in each graphic—both by increasing the amount of
variables or data points in a chart and by eliminating the use of what he has coined chartjunk. Chartjunk, according to
Tufte, is anything included in a graph that is not information, including ornamentation or thick, gaudy arrows.
Tufte also invented the sparkline, a time series chart with all axes removed and only the trendline remaining to
show historic variations of a data point without concern for exact context. Sparklines are intended to be small enough
to place in line with a body of text, similar in size to the surrounding characters, and to show the recent or historic
trend of whatever the context of the text is.
Why Data Visualization?
In William Playfair’s introduction to the Commercial and Political Atlas, he rationalizes that just as algebra is the
abbreviated shorthand for arithmetic, so are charts a way to “abbreviate and facilitate the modes of conveying
information from one person to another.” Almost 300 years later, this principle remains the same.
Data visualizations are a universal way to present complex and varied amounts of information, as we saw in our
opening example with the quarterly earnings report. They are also powerful ways to tell a story with data.
Imagine you have your Apache logs in front of you, with thousands of lines all resembling the following:
127.0.0.1 - - [10/Dec/2012:10:39:11 +0300] GET / HTTP/1.1 200 468 - Mozilla/5.0 (X11; U;
Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)
127.0.0.1 - - [10/Dec/2012:10:39:11 +0300] GET /favicon.ico HTTP/1.1 200 766 - Mozilla/5.0
(X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)
Among other things, we see IP address, date, requested resource, and client user agent. Now imagine this
repeated thousands of times—so many times that your eyes kind of glaze over because each line so closely resembles
the ones around it that it’s hard to discern where each line ends, let alone what cumulative trends exist within.
By using some analysis and visualization tools such as R, or even a commercial product such as Splunk, we can
artfully pull out all kinds of meaningful and interesting stories out of this log, from how often certain HTTP errors occur
and for which resources, to what our most widely used URLs are, to what the geographic distribution of our user base is.
This is just our Apache access log. Imagine casting a wider net, pulling in release information, bugs and
production incidents. What insights we could gather about what we do: from how our velocity impacts our defect
density to how our bugs are distributed across our feature sets. And what better way to communicate those findings
and tell those stories than through a universally digestible medium, like data visualizations?
The point of this book is to explore how we as developers can leverage this practice and medium as part of
continual improvement—both to identify and quantify our successes and opportunities for improvements, and more
effectively communicate our learning and our progress.
Tools
There are a number of excellent tools, environments, and libraries that we can use both to analyze and visualize our
data. The next two sections describe them.](https://image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-17-2048.jpg)
![Chapter 1 ■ Background
15
Creating data visualizations involves four core steps:
1. Identify a problem.
2. Gather the data.
3. Analyze the data.
4. Visualize the data.
Let’s walk through each step in the process and re-create one of the previous charts to demonstrate the process.
Identify a Problem
The very first step is to identify a problem we want to solve. This can be almost anything—from something as
profound and wide-reaching as figuring out why your bug backlog doesn’t seem to go down and stay down, to seeing
what feature releases over a given period in time caused the most production incidents, and why.
For our example, let’s re-create Figure 1-5 and try to quantify the interest in data visualization over time as
represented by the number of New York Times articles on the subject.
Gather Data
We have an idea of what we want to investigate, so let’s dig in. If you are trying to solve a problem or tell a story around
your own product, you would of course start with your own data—maybe your Apache logs, maybe your bug backlog,
maybe exports from your project tracking software.
Note
■
■ If you are focusing on gathering metrics around your product and you don’t already have data handy, you need to
invest in instrumentation.There are many ways to do this, usually by putting logging in your code.At the very least, you want to
log error states and monitor those, but you may want to expand the scope of what you track to include for
debugging purposes
while still respecting both your user’s privacy and your company’s privacy policy. In my book, Pro JavaScript
Performance:
Monitoring and Visualization, I explore ways to track and visualize web and runtime performance.
One important aspect of data gathering is deciding which format your data should be in (if you're lucky) or discovering
which format your data is available in. We’ll next be looking at some of the common data formats in use today.
JSON is an acronym that stands for JavaScript Object Notation. As you probably know, it is essentially a way to
send data as serialized JavaScript objects. We format JSON as follows:
[object]{
[attribute]: [value],
[method] : function(){},
[array]: [item, item]
}
Another way to transfer data is in XML format. XML has an expected syntax, in which elements can have attributes,
which have values, values are always in quotes, and every element must have a closing element. XML looks like this:
parent attribute=value
child attribute=valuenode data/child
/parent
Generally we can expect APIs to return XML or JSON to us, and our preference is usually JSON because as we can
see it is a much more lightweight option just in sheer amount of characters used.](https://image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-22-2048.jpg)
![Chapter 1 ■ Background
17
byline: By AUTHOR,
date: 20121011,
title: TITLE,
url: http://www.nytimes.com/foo.html
}, {
body: BODY COPY,
byline: By AUTHOR,
date: 20121021,
title: TITLE,
url: http://www.nytimes.com/bar.html
}
],
tokens: [
JavaScript
],
total: 2
}
Looking at the high-level JSON structure, we see an attribute named offset, an array named results, an array
named tokens, and another attribute named total. The offset variable is for pagination (what page full of results
we are starting with). The total variable is just what it sounds like: the number of results that are returned for our
query. It’s the results array that we really care about; it is an array of objects, each of which corresponds to an article.
The article objects have attributes named body, byline, date, title, and url.
We now have data that we can begin to look at. That takes us to our next step in the process, analyzing our data.
DATA SCRUBBING
There is often a hidden step here, one that anyone who’s dealt with data knows about: scrubbing the data. Often
the data is either not formatted exactly as we need it or, in even worse cases, it is dirty or incomplete.
In the best-case scenario in which your data just needs to be reformatted or even concatenated, go ahead and do
that, but be sure to not lose the integrity of the data.
Dirty data has fields out of order, fields with obviously bad information in them—think strings in ZIP codes—or
gaps in the data. If your data is dirty, you have several choices:
You could drop the rows in question, but that can harm the integrity of the data—a good example
•
is if you are creating a histogram removing rows could change the distribution and change what
your results will be.
The better alternative is to reach out to whoever administers the source of your data and try and
•
get a better version if it exists.
Whatever the case, if data is dirty or it just needs to be reformatted to be able to be imported into R, expect to
have to scrub your data at some point before you begin your analysis.
Analyze Data
Having data is great, but what does it mean? We determine it through analysis.
Analysis is the most crucial piece of creating data visualizations. It’s only through analysis that we can understand
our data, and it is only through understanding it that we can craft our story to share with others.](https://image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-24-2048.jpg)
![Chapter 1 ■ Background
19
if(length(author) 1){
author - unknown
}
datestamp -paste(month, /, day, /, year, sep=)
datestamp - as.Date(datestamp,%m/%d/%Y)
newrow - data.frame(datestamp, author, title, stringsAsFactors=FALSE, check.rows=FALSE)
df - rbind(df, newrow)
}
rownames(df) - df$datestamp
Note that our example assumes that the data set returned has unique date values. If you get errors with this, you
may need to scrub your returned data set to purge any duplicate rows.
Once our data frame is populated, we can start to do some analysis on the data. Let’s start out by pulling just the
year from every entry, and quickly making a stem and leaf plot to see the shape of the data.
Note
■
■ John Tukey created the stem and leaf plot in his seminal work, Exploratory Data Analysis. Stem and leaf plots
are quick, high-level ways to see the shape of data, much like a histogram. In the stem and leaf plot, we construct the
“stem” column on the left and the “leaf” column on the right. The stem consists of the most significant unique elements
in a result set. The leaf consists of the remainder of the values associated with each stem. In our stem and leaf plot below,
the years are our stem and R shows zeroes for each row associated with a given year. Something else to note is that
often alternating sequential rows are combined into a single row, in the interest of having a more concise visualization.
First, we will create a new variable to hold the year information:
yearlist - as.POSIXlt(df$datestamp)$year+1900
If we inspect this variable, we see that it looks something like this:
yearlist
[1] 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2011 2011 2011 2011 2011 2011
2011 2011 2011 2011 2011 2011 2011 2011 2011 2011
[30] 2011 2011 2011 2011 2010 2010 2010 2010 2010 2010 2010 2010 2010 2010 2009 2009 2009 2009 2009
2009 2009 2008 2008 2008 2007 2007 2007 2007 2006
[59] 2006 2006 2006 2005 2005 2005 2005 2005 2005 2004 2003 2003 2003 2002 2002 2002 2002 2001 2001
2000 2000 2000 2000 2000 2000 1999 1999 1999 1999
[88] 1999 1999 1998 1998 1998 1997 1997 1996 1996 1995 1995 1995 1993 1993 1993 1993 1992 1991 1991
1991 1990 1990 1990 1990 1989 1989 1989 1988 1988
[117] 1988 1986 1985 1985 1985 1984 1982 1982 1981
That’s great, that’s exactly what we want: a year to represent every article returned. Next let’s create the stem and
leaf plot:
stem(yearlist)
1980 | 0
1982 | 00
1984 | 0000
1986 | 0
1988 | 000000](https://image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-26-2048.jpg)
![Chapter 1 ■ Background
20
1990 | 0000000
1992 | 00000
1994 | 000
1996 | 0000
1998 | 000000000
2000 | 00000000
2002 | 0000000
2004 | 0000000
2006 | 00000000
2008 | 0000000000
2010 | 000000000000000000000000000000
2012 | 0000000000000
Very interesting. We see a gradual build with some dips in the mid-1990s, another gradual build with another dip
in the mid-2000s and a strong explosion since 2010 (the stem and leaf plot groups years together in twos).
Looking at that, my mind starts to envision a story building about a subject growing in popularity. But what
about the authors of these articles? Maybe they are the result of one or two very interested authors that have quite
a bit to say on the subject.
Let’s explore that idea and take a look at the author data that we parsed out. Let’s look at just the unique authors
from our data frame:
length(unique(df$author))
[1] 81
We see that there are 81 unique authors or combination of authors for these articles! Just out of curiosity, let’s take
a look at the breakdown by author for each article. Let’s quickly create a bar chart to see the overall shape of the data
(the bar chart is shown in Figure 1-17):
plot(table(df$author), axes=FALSE)
Figure 1-17. Bar chart of number of articles by author to quickly visualize](https://image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-27-2048.jpg)
!['I come not here to talk. You know too well
The story of our thralldom.—'
Here he made a gesture with both hands, which Jones declared
looked like a turtle trying to crawl up hill.
'We are slaves.'
A solemn pause.
'The bright sun rises to his course, and lights
A race of slaves; he sets, and his last beam
Falls on a slave.
Friends, Romans, countrymen—'
I say, interrupted Seth Williams, in an audible whisper, nudging the
orator, s'pose you leave Rome, and come down to our present age.
Give us something about the new Confederacy.
That's just what I am coming to, said Mr. Diggs, and I hope you
will not interrupt me again. After a short pause he resumed:
It is no common cause which brings us here to-night. Tyrants and
traitors are abroad in the land. A gigantic foe is invading the fair soil
of Virginia, and we are here to protect our firesides. All law writers,
from Blackstone down, agree that all men should protect their
homes. Now, fellow-citizens, remember our forefathers all fought,
and bled, and died for this glorious Union. [Applause.]
Touch lightly on that, whispered Jones.
I repeat said Mr. Diggs, that Washington was the greatest man
that ever lived. And now, grown eloquent and excited, he mounted
a bench and whipped his left hand under the tails of his coat, while
he waved his right in vehement gesture.](https://image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-40-2048.jpg)
![Chapter 1 ■ Background
10
This chart was created in response to the Fukushima Daiichi nuclear disaster of 2011, and sought to clear up
misinformation and misunderstanding of comparisons being made around the disaster. It did this by demonstrating the
differences in scale for the amount of radiation from sources such as other people or a banana, up to what a fatal dose of
radiation ultimately would be—how all that compared to spending just ten minutes near the Chernobyl meltdown.
Over the last quarter of a century, Edward Tufte, author and professor emeritus at Yale University, has been
working to raise the bar of information graphics. He published groundbreaking books detailing the history of data
visualization, tracing its roots even further back than Playfair, to the beginnings of cartography. Among his principles
is the idea to maximize the amount of information included in each graphic—both by increasing the amount of
variables or data points in a chart and by eliminating the use of what he has coined chartjunk. Chartjunk, according to
Tufte, is anything included in a graph that is not information, including ornamentation or thick, gaudy arrows.
Tufte also invented the sparkline, a time series chart with all axes removed and only the trendline remaining to
show historic variations of a data point without concern for exact context. Sparklines are intended to be small enough
to place in line with a body of text, similar in size to the surrounding characters, and to show the recent or historic
trend of whatever the context of the text is.
Why Data Visualization?
In William Playfair’s introduction to the Commercial and Political Atlas, he rationalizes that just as algebra is the
abbreviated shorthand for arithmetic, so are charts a way to “abbreviate and facilitate the modes of conveying
information from one person to another.” Almost 300 years later, this principle remains the same.
Data visualizations are a universal way to present complex and varied amounts of information, as we saw in our
opening example with the quarterly earnings report. They are also powerful ways to tell a story with data.
Imagine you have your Apache logs in front of you, with thousands of lines all resembling the following:
127.0.0.1 - - [10/Dec/2012:10:39:11 +0300] GET / HTTP/1.1 200 468 - Mozilla/5.0 (X11; U;
Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)
127.0.0.1 - - [10/Dec/2012:10:39:11 +0300] GET /favicon.ico HTTP/1.1 200 766 - Mozilla/5.0
(X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty)
Among other things, we see IP address, date, requested resource, and client user agent. Now imagine this
repeated thousands of times—so many times that your eyes kind of glaze over because each line so closely resembles
the ones around it that it’s hard to discern where each line ends, let alone what cumulative trends exist within.
By using some analysis and visualization tools such as R, or even a commercial product such as Splunk, we can
artfully pull out all kinds of meaningful and interesting stories out of this log, from how often certain HTTP errors occur
and for which resources, to what our most widely used URLs are, to what the geographic distribution of our user base is.
This is just our Apache access log. Imagine casting a wider net, pulling in release information, bugs and
production incidents. What insights we could gather about what we do: from how our velocity impacts our defect
density to how our bugs are distributed across our feature sets. And what better way to communicate those findings
and tell those stories than through a universally digestible medium, like data visualizations?
The point of this book is to explore how we as developers can leverage this practice and medium as part of
continual improvement—both to identify and quantify our successes and opportunities for improvements, and more
effectively communicate our learning and our progress.
Tools
There are a number of excellent tools, environments, and libraries that we can use both to analyze and visualize our
data. The next two sections describe them.](https://crownmelresort.com/image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-17-2048.jpg)
![Chapter 1 ■ Background
15
Creating data visualizations involves four core steps:
1. Identify a problem.
2. Gather the data.
3. Analyze the data.
4. Visualize the data.
Let’s walk through each step in the process and re-create one of the previous charts to demonstrate the process.
Identify a Problem
The very first step is to identify a problem we want to solve. This can be almost anything—from something as
profound and wide-reaching as figuring out why your bug backlog doesn’t seem to go down and stay down, to seeing
what feature releases over a given period in time caused the most production incidents, and why.
For our example, let’s re-create Figure 1-5 and try to quantify the interest in data visualization over time as
represented by the number of New York Times articles on the subject.
Gather Data
We have an idea of what we want to investigate, so let’s dig in. If you are trying to solve a problem or tell a story around
your own product, you would of course start with your own data—maybe your Apache logs, maybe your bug backlog,
maybe exports from your project tracking software.
Note
■
■ If you are focusing on gathering metrics around your product and you don’t already have data handy, you need to
invest in instrumentation.There are many ways to do this, usually by putting logging in your code.At the very least, you want to
log error states and monitor those, but you may want to expand the scope of what you track to include for
debugging purposes
while still respecting both your user’s privacy and your company’s privacy policy. In my book, Pro JavaScript
Performance:
Monitoring and Visualization, I explore ways to track and visualize web and runtime performance.
One important aspect of data gathering is deciding which format your data should be in (if you're lucky) or discovering
which format your data is available in. We’ll next be looking at some of the common data formats in use today.
JSON is an acronym that stands for JavaScript Object Notation. As you probably know, it is essentially a way to
send data as serialized JavaScript objects. We format JSON as follows:
[object]{
[attribute]: [value],
[method] : function(){},
[array]: [item, item]
}
Another way to transfer data is in XML format. XML has an expected syntax, in which elements can have attributes,
which have values, values are always in quotes, and every element must have a closing element. XML looks like this:
parent attribute=value
child attribute=valuenode data/child
/parent
Generally we can expect APIs to return XML or JSON to us, and our preference is usually JSON because as we can
see it is a much more lightweight option just in sheer amount of characters used.](https://crownmelresort.com/image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-22-2048.jpg)
![Chapter 1 ■ Background
17
byline: By AUTHOR,
date: 20121011,
title: TITLE,
url: http://www.nytimes.com/foo.html
}, {
body: BODY COPY,
byline: By AUTHOR,
date: 20121021,
title: TITLE,
url: http://www.nytimes.com/bar.html
}
],
tokens: [
JavaScript
],
total: 2
}
Looking at the high-level JSON structure, we see an attribute named offset, an array named results, an array
named tokens, and another attribute named total. The offset variable is for pagination (what page full of results
we are starting with). The total variable is just what it sounds like: the number of results that are returned for our
query. It’s the results array that we really care about; it is an array of objects, each of which corresponds to an article.
The article objects have attributes named body, byline, date, title, and url.
We now have data that we can begin to look at. That takes us to our next step in the process, analyzing our data.
DATA SCRUBBING
There is often a hidden step here, one that anyone who’s dealt with data knows about: scrubbing the data. Often
the data is either not formatted exactly as we need it or, in even worse cases, it is dirty or incomplete.
In the best-case scenario in which your data just needs to be reformatted or even concatenated, go ahead and do
that, but be sure to not lose the integrity of the data.
Dirty data has fields out of order, fields with obviously bad information in them—think strings in ZIP codes—or
gaps in the data. If your data is dirty, you have several choices:
You could drop the rows in question, but that can harm the integrity of the data—a good example
•
is if you are creating a histogram removing rows could change the distribution and change what
your results will be.
The better alternative is to reach out to whoever administers the source of your data and try and
•
get a better version if it exists.
Whatever the case, if data is dirty or it just needs to be reformatted to be able to be imported into R, expect to
have to scrub your data at some point before you begin your analysis.
Analyze Data
Having data is great, but what does it mean? We determine it through analysis.
Analysis is the most crucial piece of creating data visualizations. It’s only through analysis that we can understand
our data, and it is only through understanding it that we can craft our story to share with others.](https://crownmelresort.com/image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-24-2048.jpg)
![Chapter 1 ■ Background
19
if(length(author) 1){
author - unknown
}
datestamp -paste(month, /, day, /, year, sep=)
datestamp - as.Date(datestamp,%m/%d/%Y)
newrow - data.frame(datestamp, author, title, stringsAsFactors=FALSE, check.rows=FALSE)
df - rbind(df, newrow)
}
rownames(df) - df$datestamp
Note that our example assumes that the data set returned has unique date values. If you get errors with this, you
may need to scrub your returned data set to purge any duplicate rows.
Once our data frame is populated, we can start to do some analysis on the data. Let’s start out by pulling just the
year from every entry, and quickly making a stem and leaf plot to see the shape of the data.
Note
■
■ John Tukey created the stem and leaf plot in his seminal work, Exploratory Data Analysis. Stem and leaf plots
are quick, high-level ways to see the shape of data, much like a histogram. In the stem and leaf plot, we construct the
“stem” column on the left and the “leaf” column on the right. The stem consists of the most significant unique elements
in a result set. The leaf consists of the remainder of the values associated with each stem. In our stem and leaf plot below,
the years are our stem and R shows zeroes for each row associated with a given year. Something else to note is that
often alternating sequential rows are combined into a single row, in the interest of having a more concise visualization.
First, we will create a new variable to hold the year information:
yearlist - as.POSIXlt(df$datestamp)$year+1900
If we inspect this variable, we see that it looks something like this:
yearlist
[1] 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2011 2011 2011 2011 2011 2011
2011 2011 2011 2011 2011 2011 2011 2011 2011 2011
[30] 2011 2011 2011 2011 2010 2010 2010 2010 2010 2010 2010 2010 2010 2010 2009 2009 2009 2009 2009
2009 2009 2008 2008 2008 2007 2007 2007 2007 2006
[59] 2006 2006 2006 2005 2005 2005 2005 2005 2005 2004 2003 2003 2003 2002 2002 2002 2002 2001 2001
2000 2000 2000 2000 2000 2000 1999 1999 1999 1999
[88] 1999 1999 1998 1998 1998 1997 1997 1996 1996 1995 1995 1995 1993 1993 1993 1993 1992 1991 1991
1991 1990 1990 1990 1990 1989 1989 1989 1988 1988
[117] 1988 1986 1985 1985 1985 1984 1982 1982 1981
That’s great, that’s exactly what we want: a year to represent every article returned. Next let’s create the stem and
leaf plot:
stem(yearlist)
1980 | 0
1982 | 00
1984 | 0000
1986 | 0
1988 | 000000](https://crownmelresort.com/image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-26-2048.jpg)
![Chapter 1 ■ Background
20
1990 | 0000000
1992 | 00000
1994 | 000
1996 | 0000
1998 | 000000000
2000 | 00000000
2002 | 0000000
2004 | 0000000
2006 | 00000000
2008 | 0000000000
2010 | 000000000000000000000000000000
2012 | 0000000000000
Very interesting. We see a gradual build with some dips in the mid-1990s, another gradual build with another dip
in the mid-2000s and a strong explosion since 2010 (the stem and leaf plot groups years together in twos).
Looking at that, my mind starts to envision a story building about a subject growing in popularity. But what
about the authors of these articles? Maybe they are the result of one or two very interested authors that have quite
a bit to say on the subject.
Let’s explore that idea and take a look at the author data that we parsed out. Let’s look at just the unique authors
from our data frame:
length(unique(df$author))
[1] 81
We see that there are 81 unique authors or combination of authors for these articles! Just out of curiosity, let’s take
a look at the breakdown by author for each article. Let’s quickly create a bar chart to see the overall shape of the data
(the bar chart is shown in Figure 1-17):
plot(table(df$author), axes=FALSE)
Figure 1-17. Bar chart of number of articles by author to quickly visualize](https://crownmelresort.com/image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-27-2048.jpg)
!['I come not here to talk. You know too well
The story of our thralldom.—'
Here he made a gesture with both hands, which Jones declared
looked like a turtle trying to crawl up hill.
'We are slaves.'
A solemn pause.
'The bright sun rises to his course, and lights
A race of slaves; he sets, and his last beam
Falls on a slave.
Friends, Romans, countrymen—'
I say, interrupted Seth Williams, in an audible whisper, nudging the
orator, s'pose you leave Rome, and come down to our present age.
Give us something about the new Confederacy.
That's just what I am coming to, said Mr. Diggs, and I hope you
will not interrupt me again. After a short pause he resumed:
It is no common cause which brings us here to-night. Tyrants and
traitors are abroad in the land. A gigantic foe is invading the fair soil
of Virginia, and we are here to protect our firesides. All law writers,
from Blackstone down, agree that all men should protect their
homes. Now, fellow-citizens, remember our forefathers all fought,
and bled, and died for this glorious Union. [Applause.]
Touch lightly on that, whispered Jones.
I repeat said Mr. Diggs, that Washington was the greatest man
that ever lived. And now, grown eloquent and excited, he mounted
a bench and whipped his left hand under the tails of his coat, while
he waved his right in vehement gesture.](https://crownmelresort.com/image.slidesharecdn.com/2969-250516103138-2bdf523a/75/Pro-Data-Visualization-using-R-and-JavaScript-1st-Edition-Tom-Barker-40-2048.jpg)
Pro Data Visualization using R and JavaScript 1st Edition Tom Barker
- 1. Pro Data Visualizationusing R and JavaScript 1st Edition Tom Barker download https://ebookname.com/product/pro-data-visualization-using-r-and- javascript-1st-edition-tom-barker/ Get Instant Ebook Downloads – Browse at https://ebookname.com
- 2. Instant digital products(PDF, ePub, MOBI) available Download now and explore formats that suit you... A Primer in Biological Data Analysis and Visualization Using R Gregg Hartvigsen https://ebookname.com/product/a-primer-in-biological-data- analysis-and-visualization-using-r-gregg-hartvigsen/ Visual Data Mining Techniques and Tools for Data Visualization and Mining 1st Edition Tom Soukup https://ebookname.com/product/visual-data-mining-techniques-and- tools-for-data-visualization-and-mining-1st-edition-tom-soukup/ Data Visualization with Python and JavaScript Scrape Clean Explore Transform Your Data 1st Edition Kyran Dale https://ebookname.com/product/data-visualization-with-python-and- javascript-scrape-clean-explore-transform-your-data-1st-edition- kyran-dale/ Merciless Zebra Romantic Suspense Mary Burton https://ebookname.com/product/merciless-zebra-romantic-suspense- mary-burton/
- 3. Milestone Documents inAmerican History Exploring the Primary Sources That Shaped America 1st Edition Paul Finkelman https://ebookname.com/product/milestone-documents-in-american- history-exploring-the-primary-sources-that-shaped-america-1st- edition-paul-finkelman/ Knowledge Is Pleasure Florence Ayscough in Shanghai 1st Edition Lindsay Shen https://ebookname.com/product/knowledge-is-pleasure-florence- ayscough-in-shanghai-1st-edition-lindsay-shen/ Transcription Factors 1st Edition Joseph Locker https://ebookname.com/product/transcription-factors-1st-edition- joseph-locker/ Handbook of SCADA control systems security 1st Edition Radvanovsky https://ebookname.com/product/handbook-of-scada-control-systems- security-1st-edition-radvanovsky/ Jini and JavaSpaces Application Development 1st Edition Robert Flenner https://ebookname.com/product/jini-and-javaspaces-application- development-1st-edition-robert-flenner/
- 4. Quality Management 1stEdition Graeme Knowles https://ebookname.com/product/quality-management-1st-edition- graeme-knowles/
- 6. For your convenienceApress has placed some of the front matter material after the index. Please use the Bookmarks and Contents at a Glance links to access them.
- 7. v Contents at aGlance About the Author���������������������������������������������������������������������������������������������������������������xiii About the Technical Reviewer��������������������������������������������������������������������������������������������xv Acknowledgments������������������������������������������������������������������������������������������������������������xvii Chapter 1: Background ■ ■ ������������������������������������������������������������������������������������������������������1 Chapter 2: R Language Primer ■ ■ ����������������������������������������������������������������������������������������25 Chapter 3: A Deeper Dive into R ■ ■ ��������������������������������������������������������������������������������������47 Chapter 4: Data Visualization with D3 ■ ■ �����������������������������������������������������������������������������65 Chapter 5: Visualizing Spatial Data from Access Logs ■ ■ ����������������������������������������������������85 Chapter 6: Visualizing Data Over Time ■ ■ ��������������������������������������������������������������������������111 Chapter 7: Bar Charts ■ ■ ����������������������������������������������������������������������������������������������������133 Chapter 8: Correlation Analysis with Scatter Plots ■ ■ �������������������������������������������������������157 Chapter 9: Visualizing the Balance of Delivery and Quality with ■ ■ Parallel Coordinates������������������������������������������������������������������������������������������������������177 Index���������������������������������������������������������������������������������������������������������������������������������193
- 8. 1 Chapter 1 Background There isa new concept emerging in the field of web development: using data visualizations as communication tools. This concept is something that is already well established in other fields and departments. At the company where you work, your finance department probably uses data visualizations to represent fiscal information both internally and externally; just take a look at the quarterly earnings reports for almost any publicly traded company. They are full of charts to show revenue by quarter, or year over year earnings, or a plethora of other historic financial data. All are designed to show lots and lots of data points, potentially pages and pages of data points, in a single easily digestible graphic. Compare the bar chart in Google’s quarterly earnings report from back in 2007 (see Figure 1-1) to a subset of the data it is based on in tabular format (see Figure 1-2). Figure 1-1. Google Q4 2007 quarterly revenue shown in a bar chart
- 9. Chapter 1 ■Background 2 The bar chart is imminently more readable. We can clearly see by the shape of it that earnings are up and have been steadily going up each quarter. By the color-coding, we can see the sources of the earnings; and with the annotations, we can see both the precise numbers that those color-coding represent and what the year over year percentages are. With the tabular data, you have to read labels on the left, line up the data on the right with those labels, do your own aggregation and comparison, and draw your own conclusions. There is a lot more upfront work needed to take in the tabular data, and there exists the very real possibility of your audience either not understanding the data (thus creating their own incorrect story around the data) or tuning out completely because of the sheer amount of work needed to take in the information. It’s not just the Finance department that uses visualizations to communicate dense amounts of data. Maybe your Operations department uses charts to communicate server uptime, or your Customer Support department uses graphs to show call volume. Whatever the case, it’s about time Engineering and Web Development got on board with this. As a department, group, and industry we have a huge amount of relevant data that is important for us to first be aware of so that we can refine and improve what we do; but also to communicate out to our stakeholders, to demonstrate our successes or validate resource needs, or to plan tactical roadmaps for the coming year. Before we can do this, we need to understand what we are doing. We need to understand what data visualizations are, a general idea of their history, when to use them, and how to use them both technically and ethically. What Is Data Visualization? OK, so what exactly is data visualization? Data visualization is the art and practice of gathering, analyzing, and graphically representing empirical information. They are sometimes called information graphics, or even just charts and graphs. Whatever you call it, the goal of visualizing data is to tell the story in the data. Telling the story is predicated on understanding the data at a very deep level, and gathering insight from comparisons of data points in the numbers. There exists syntax for crafting data visualizations, patterns in the form of charts that have an immediately known context. We devote a chapter to each of the significant chart types later in the book. Time Series Charts Time series charts show changes over time. See Figure 1-3 for a time series chart that shows the weighted popularity of the keyword “Data Visualization” from Google Trends (http://www.google.com/trends/). Figure 1-2. Similar earnings data in tabular form
- 10. Chapter 1 ■Background 3 Note that the vertical y axis shows a sequence of numbers that increment by 20 up to 100. These numbers represent the weighted search volume, where 100 is the peak search volume for our term. On the horizontal x axis, we see years going from 2007 to 2012. The line in the chart represents both axes, the given search volume for each date. From just this small sample size, we can see that the term has more than tripled in popularity, from a low of 29 in the beginning of 2007 up to the ceiling of 100 by the end of 2012. Bar Charts Bar charts show comparisons of data points. See Figure 1-4 for a bar chart that demonstrates the search volume by country for the keyword “Data Visualization,” the data for which is also sourced from Google Trends. Figure 1-3. Time series of weighted trend for the keyword “Data Visualization” from Google Trends Search Volume for Keyword ‘Data Visualization’ by Region from Google Trends Spain France Germany China United Kingdom Netherlands Australia Canada India United States 0 20 40 60 80 100 Figure 1-4. Google Trends breakdown of search volume by region for keyword “Data Visualization”
- 11. Chapter 1 ■Background 4 We can see the names of the countries on the y axis and the normalized search volume, from 0 to 100, on the x axis. Notice, though, that no time measure is given. Does this chart represent data for a day, a month, or a year? Also note that we have no context for what the unit of measure is. I highlight these points not to answer them but to demonstrate the limitations and pitfalls of this particular chart type. We must always be aware that our audience does not bring the same experience and context that we bring, so we must strive to make the stories in our visualizations as self evident as possible. Histograms Histograms are a type of bar chart used to show the distribution of data or how often groups of information appear in the data. See Figure 1-5 for a histogram that shows how many articles the New York Times published each year, from 1980 to 2012, that related in some way to the subject of data visualization. We can see from the chart that the subject has been ramping up in frequency since 2009. 1980 1985 1990 1995 2000 2005 2010 Year Distribution of Articles about Data Visualization by the NY Times Frequency 20 15 10 5 0 Figure 1-5. Histogram showing distribution of NY Times articles about data visualization
- 12. Chapter 1 ■Background 5 In this example, the states with the darker shades indicate a greater interest in the search term. (This data also is derived from Google Trends, for which interest is demonstrated by how frequently the term “Data Visualization” is searched for on Google.) Scatter Plots Like bar charts, scatter plots are used to compare data, but specifically to suggest correlations in the data, or where the data may be dependent or related in some way. See Figure 1-7, in which we use data from Google Correlate, (http://www.google.com/trends/correlate), to look for a relationship between search volume for the keyword “What is Data Visualization” and the keyword “How to Create Data Visualization.” Figure 1-6. Data map of U.S. states by interest in “Data Visualization” (data from Google Trends) Data Maps Data maps are used to show the distribution of information over a spatial region. Figure 1-6 shows a data map used to demonstrate the interest in the search term “Data Visualization” broken out by U.S. states.
- 13. Chapter 1 ■Background 6 This chart suggests a positive correlation in the data, meaning that as one term rises in popularity the other also rises. So what this chart suggests is that as more people find out about data visualization, more people want to learn how to create data visualizations. The important thing to remember about correlation is that it does not suggest a direct cause—correlation is not causation. History If we’re talking about the history of data visualization, the modern conception of data visualization largely started with William Playfair. William Playfair was, among other things, an engineer, an accountant, a banker, and an all-around Renaissance man who single handedly created the time series chart, the bar chart, and the bubble chart. Playfair’s charts were published in the late eighteenth century into the early nineteenth century. He was very aware that his innovations were the first of their kind, at least in the realm of communicating statistical information, and he spent a good amount of space in his books describing how to make the mental leap to seeing bars and lines as representing physical things like money. Playfair is best known for two of his books: the Commercial and Political Atlas and the Statistical Breviary. The Commercial and Political Atlas was published in 1786 and focused on different aspects of economic data from national debt, to trade figures, and even military spending. It also featured the first printed time series graph and bar chart. Figure 1-7. Scatter plot examining the correlation between search volume for terms related to “Data Visualization” , “How to Create” and “What is”
- 14. Chapter 1 ■Background 7 His Statistical Breviary focused on statistical information around the resources of the major European countries of the time and introduced the bubble chart. Playfair had several goals with his charts, among them perhaps stirring controversy, commenting on the diminishing spending power of the working class, and even demonstrating the balance of favor in the import and export figures of the British Empire, but ultimately his most wide-reaching goal was to communicate complex statistical information in an easily digested, universally understood format. Note ■ ■ Both books are back in print relatively recently, thanks to Howard Wainer, Ian Spence, and Cambridge University Press. Playfair had several contemporaries, including Dr. John Snow, who made my personal favorite chart: the cholera map. The cholera map is everything an informational graphic should be: it was simple to read; it was informative; and, most importantly, it solved a real problem. The cholera map is a data map that outlined the location of all the diagnosed cases of cholera in the outbreak of London 1854 (see Figure 1-8). The shaded areas are recorded deaths from cholera, and the shaded circles on the map are water pumps. From careful inspection, the recorded deaths seemed to radiate out from the water pump on Broad Street. Figure 1-8. John Snow’s cholera map
- 15. Chapter 1 ■Background 8 Dr. Snow had the Broad Street water pump closed, and the outbreak ended. Beautiful, concise, and logical. Another historically significant information graphic is the Diagram of the Causes of Mortality in the Army in the East, by Florence Nightingale and William Farr. This chart is shown in Figure 1-9. Figure 1-9. Florence Nightingale and William Farr’s Diagram of the Causes of Mortality in the Army in the East Nightingale and Farr created this chart in 1856 to demonstrate the relative number of preventable deaths and, at a higher level, to improve the sanitary conditions of military installations. Note that the Nightingale and Farr visualization is a stylized pie chart. Pie charts are generally a circle representing the entirety of a given data set with slices of the circle representing percentages of a whole. The usefulness of pie charts is sometimes debated because it can be argued that it is harder to discern the difference in value between angles than it is to determine the length of a bar or the placement of a line against Cartesian coordinates. Nightingale seemingly avoids this pitfall by having not just the angle of the wedge hold value but by also altering the relative size of the slices so they eschew the confines of the containing circle and represent relative value. All the above examples had specific goals or problems that they were trying to solve. Note ■ ■ A rich comprehensive history is beyond the scope of this book, but if you are interested in a thoughtful, incredibly researched analysis, be sure to read Edward Tufte’s The Visual Display of Quantitative Information. Modern Landscape Data visualization is in the midst of a modern revitalization due in large part to the proliferation of cheap storage space to store logs, and free and open source tools to analyze and chart the information in these logs.
- 16. Chapter 1 ■Background 9 From a consumption and appreciation perspective, there are websites that are dedicated to studying and talking about information graphics. There are generalized sites such as FlowingData that both aggregate and discuss data visualizations from around the web, from astrophysics timelines to mock visualizations used on the floor of Congress. The mission statement from the FlowingData About page (http://flowingdata.com/about/) is appropriately the following: “FlowingData explores how designers, statisticians, and computer scientists use data to understand ourselves better—mainly through data visualization.” There are more specialized sites such as quantifiedself.com that are focused on gathering and visualizing information about oneself. There are even web comics about data visualization, the quintessential one being xkcd.com, run by Randall Munroe. One of the most famous and topical visualizations that Randall has created thus far is the Radiation Dose Chart. We can see the Radiation Dose Chart in Figure 1-10 (it is available in high resolution here: http://xkcd.com/radiation/). Figure 1-10. Radiation Dose Chart, by Randall Munroe. Note that the range in scale being represented in this visualization as a single block in one chart is exploded to show an entirely new microcosm of context and information. This pattern is repeated over and over again to show an incredible depth of information
- 17. Chapter 1 ■Background 10 This chart was created in response to the Fukushima Daiichi nuclear disaster of 2011, and sought to clear up misinformation and misunderstanding of comparisons being made around the disaster. It did this by demonstrating the differences in scale for the amount of radiation from sources such as other people or a banana, up to what a fatal dose of radiation ultimately would be—how all that compared to spending just ten minutes near the Chernobyl meltdown. Over the last quarter of a century, Edward Tufte, author and professor emeritus at Yale University, has been working to raise the bar of information graphics. He published groundbreaking books detailing the history of data visualization, tracing its roots even further back than Playfair, to the beginnings of cartography. Among his principles is the idea to maximize the amount of information included in each graphic—both by increasing the amount of variables or data points in a chart and by eliminating the use of what he has coined chartjunk. Chartjunk, according to Tufte, is anything included in a graph that is not information, including ornamentation or thick, gaudy arrows. Tufte also invented the sparkline, a time series chart with all axes removed and only the trendline remaining to show historic variations of a data point without concern for exact context. Sparklines are intended to be small enough to place in line with a body of text, similar in size to the surrounding characters, and to show the recent or historic trend of whatever the context of the text is. Why Data Visualization? In William Playfair’s introduction to the Commercial and Political Atlas, he rationalizes that just as algebra is the abbreviated shorthand for arithmetic, so are charts a way to “abbreviate and facilitate the modes of conveying information from one person to another.” Almost 300 years later, this principle remains the same. Data visualizations are a universal way to present complex and varied amounts of information, as we saw in our opening example with the quarterly earnings report. They are also powerful ways to tell a story with data. Imagine you have your Apache logs in front of you, with thousands of lines all resembling the following: 127.0.0.1 - - [10/Dec/2012:10:39:11 +0300] GET / HTTP/1.1 200 468 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty) 127.0.0.1 - - [10/Dec/2012:10:39:11 +0300] GET /favicon.ico HTTP/1.1 200 766 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty) Among other things, we see IP address, date, requested resource, and client user agent. Now imagine this repeated thousands of times—so many times that your eyes kind of glaze over because each line so closely resembles the ones around it that it’s hard to discern where each line ends, let alone what cumulative trends exist within. By using some analysis and visualization tools such as R, or even a commercial product such as Splunk, we can artfully pull out all kinds of meaningful and interesting stories out of this log, from how often certain HTTP errors occur and for which resources, to what our most widely used URLs are, to what the geographic distribution of our user base is. This is just our Apache access log. Imagine casting a wider net, pulling in release information, bugs and production incidents. What insights we could gather about what we do: from how our velocity impacts our defect density to how our bugs are distributed across our feature sets. And what better way to communicate those findings and tell those stories than through a universally digestible medium, like data visualizations? The point of this book is to explore how we as developers can leverage this practice and medium as part of continual improvement—both to identify and quantify our successes and opportunities for improvements, and more effectively communicate our learning and our progress. Tools There are a number of excellent tools, environments, and libraries that we can use both to analyze and visualize our data. The next two sections describe them.
- 18. Chapter 1 ■Background 11 Languages, Environments, and Libraries The tools that are most relevant to web developers are Splunk, R, and the D3 JavaScript library. See Figure 1-11 for a comparison of interest over time for them (from Google Trends). Figure 1-11. Google Trends analysis of interest over time in Splunk, R, and D3 From the figure we can see that R has had a steady consistent amount of interest since 200; Splunk had an introduction to the chart around 2005, had a spike of interest around 2006, and had steady growth since then. As for D3, we see it just start to peak around 2011 when it was introduced and its predecessor Protovis was sunsetted. Let’s start with the tool of choice for many developers, scientists, and statisticians: the R language. We have a deep dive into the R environment and language in the next chapter, but for now it’s enough to know that it is an open source environment and language used for statistical analysis and graphical display. It is powerful, fun to use, and, best of all, it is free. Splunk has seen a tremendous steady growth in interest over the last few years—and for good reason. It is easy to use once it’s set up, scales wonderfully, supports multiple concurrent users, and puts data reporting at the fingertips of everyone. You simply set it up to consume your log files; then you can go into the Splunk dashboard and run reports on key values within those logs. Splunk creates visualizations as part of its reporting capabilities, as well as alerting. While Splunk is a commercial product, it also offers a free version, available here: http://www.splunk.com/download. D3 is a JavaScript library that allows us to craft interactive visualizations. It is the official follow-up to Protovis. Protovis was a JavaScript library created in 2009 by Stanford University’s Stanford Visualization Group. Protovis was sunsetted in 2011, and the creators unveiled D3. We explore the D3 library at length in Chapter 4. Analysis Tools Aside from the previously mentioned languages and environments, there are a number of analysis tools available online. A great hosted tool for analysis and research is Google Trends. Google Trends allows you to compare trends on search terms. It provides all kinds of great statistical information around those trends, including comparing their relative search volume (see Figure 1-12), the geographic area those trends are coming from (see Figure 1-13), and related keywords.
- 19. Chapter 1 ■Background 12 Figure 1-13. Google Trends data map showing geographic location where interest in the key words is originating Figure 1-12. Google Trends for the terms “data scientist” and “computer scientist” over time; note the interest in the term “data scientist” growing rapidly from 2011 on to match the interest in the term “computer scientist”
- 20. Chapter 1 ■Background 13 Another great tool for analysis is Wolfram|Alpha (http://wolframalpha.com). See Figure 1-14 for a screenshot of the Wolfram|Alpha homepage. Figure 1-14. Home page for Wolfram|Alpha Wolfram|Alpha is not a search engine. Search engines spider and index content. Wolfram|Alpha is instead a Question Answering (QA) engine that parses human readable sentences with natural language processing and responds with computed results. Say, for example, you want to search for the speed of light. You might go to the Wolfram|Alpha site and type in “What is the speed of light?” Remember that it uses natural language processing to parse your search query, not the keyword lookup. The results of this query can be seen in Figure 1-15. Wolfram|Alpha essentially looks up all the data it has around the speed of light and presents it in a structured, categorized fashion. You can also export the raw data for each result.
- 21. Chapter 1 ■Background 14 Figure 1-15. Wolfram|Alpha results for query What is the speed of light Process Overview So we understand what data visualization is, have a high-level understanding of the history of it and an idea of the current landscape. We’re beginning to get an inkling about how we can start to use this in our world. We know some of the tools that are available to us to facilitate the analysis and creation of our charts. Now let’s look at the process involved.
- 22. Chapter 1 ■Background 15 Creating data visualizations involves four core steps: 1. Identify a problem. 2. Gather the data. 3. Analyze the data. 4. Visualize the data. Let’s walk through each step in the process and re-create one of the previous charts to demonstrate the process. Identify a Problem The very first step is to identify a problem we want to solve. This can be almost anything—from something as profound and wide-reaching as figuring out why your bug backlog doesn’t seem to go down and stay down, to seeing what feature releases over a given period in time caused the most production incidents, and why. For our example, let’s re-create Figure 1-5 and try to quantify the interest in data visualization over time as represented by the number of New York Times articles on the subject. Gather Data We have an idea of what we want to investigate, so let’s dig in. If you are trying to solve a problem or tell a story around your own product, you would of course start with your own data—maybe your Apache logs, maybe your bug backlog, maybe exports from your project tracking software. Note ■ ■ If you are focusing on gathering metrics around your product and you don’t already have data handy, you need to invest in instrumentation.There are many ways to do this, usually by putting logging in your code.At the very least, you want to log error states and monitor those, but you may want to expand the scope of what you track to include for debugging purposes while still respecting both your user’s privacy and your company’s privacy policy. In my book, Pro JavaScript Performance: Monitoring and Visualization, I explore ways to track and visualize web and runtime performance. One important aspect of data gathering is deciding which format your data should be in (if you're lucky) or discovering which format your data is available in. We’ll next be looking at some of the common data formats in use today. JSON is an acronym that stands for JavaScript Object Notation. As you probably know, it is essentially a way to send data as serialized JavaScript objects. We format JSON as follows: [object]{ [attribute]: [value], [method] : function(){}, [array]: [item, item] } Another way to transfer data is in XML format. XML has an expected syntax, in which elements can have attributes, which have values, values are always in quotes, and every element must have a closing element. XML looks like this: parent attribute=value child attribute=valuenode data/child /parent Generally we can expect APIs to return XML or JSON to us, and our preference is usually JSON because as we can see it is a much more lightweight option just in sheer amount of characters used.
- 23. Chapter 1 ■Background 16 But if we are exporting data from an application, it most likely will be in the form of a comma separated value file, or CSV. A CSV is exactly what it sounds like: values separated by commas or some other sort of delimiter: value1,value2,value3 value4,value5,value6 For our example, we’ll use the New York Times API Tool, available at http://prototype.nytimes.com/gst/ apitool/index.html. The API Tool exposes all the APIs that the New York Times makes available, including the Article Search API, the Campaign Finance API, and the Movie Review API. All we need to do is select the Article Search API from the drop-down menu, type in our search query or the phrase that we want to search for, and click “Make Request” . This queries the API and returns the data to us, formatted as JSON. We can see the results in Figure 1-16. Figure 1-16. The NY Times API Tool We can then copy and paste the returned JSON data to our own file or we could go the extra step to get an API key so that we can query the API from our own applications. For the sake of our example, we will save the JSON data to a file that we will name jsNYTimesData. The contents of the file will be structured like so: { offset: 0, results: [ { body: BODY COPY,
- 24. Chapter 1 ■Background 17 byline: By AUTHOR, date: 20121011, title: TITLE, url: http://www.nytimes.com/foo.html }, { body: BODY COPY, byline: By AUTHOR, date: 20121021, title: TITLE, url: http://www.nytimes.com/bar.html } ], tokens: [ JavaScript ], total: 2 } Looking at the high-level JSON structure, we see an attribute named offset, an array named results, an array named tokens, and another attribute named total. The offset variable is for pagination (what page full of results we are starting with). The total variable is just what it sounds like: the number of results that are returned for our query. It’s the results array that we really care about; it is an array of objects, each of which corresponds to an article. The article objects have attributes named body, byline, date, title, and url. We now have data that we can begin to look at. That takes us to our next step in the process, analyzing our data. DATA SCRUBBING There is often a hidden step here, one that anyone who’s dealt with data knows about: scrubbing the data. Often the data is either not formatted exactly as we need it or, in even worse cases, it is dirty or incomplete. In the best-case scenario in which your data just needs to be reformatted or even concatenated, go ahead and do that, but be sure to not lose the integrity of the data. Dirty data has fields out of order, fields with obviously bad information in them—think strings in ZIP codes—or gaps in the data. If your data is dirty, you have several choices: You could drop the rows in question, but that can harm the integrity of the data—a good example • is if you are creating a histogram removing rows could change the distribution and change what your results will be. The better alternative is to reach out to whoever administers the source of your data and try and • get a better version if it exists. Whatever the case, if data is dirty or it just needs to be reformatted to be able to be imported into R, expect to have to scrub your data at some point before you begin your analysis. Analyze Data Having data is great, but what does it mean? We determine it through analysis. Analysis is the most crucial piece of creating data visualizations. It’s only through analysis that we can understand our data, and it is only through understanding it that we can craft our story to share with others.
- 25. Chapter 1 ■Background 18 To begin analysis, let’s import our data into R. Don’t worry if you aren’t completely fluent in R; we do a deep dive into the language in the next chapter. If you aren’t familiar with R yet, don’t worry about coding along with the following examples: just follow along to get an idea of what is happening and return to these examples after reading Chapters 3 and 4. Because our data is JSON, let’s use an R package called rjson. This will allow us to read in and parse JSON with the fromJSON() function: library(rjson) json_data - fromJSON(paste(readLines(jsNYTimesData.txt), collapse=)) This is great, except the data is read in as pure text, including the date information. We can’t extract information from text because obviously text has no contextual meaning outside of being raw characters. So we need to iterate through the data and parse it to more meaningful types. Let's create a data frame (an array-like data type specific to R that we talk about next chapter), loop through our json_data object; and parse year, month, and day parts out of the date attribute. Let’s also parse the author name out of the byline, and check to make sure that if the author’s name isn’t present we substitute the empty value with the string “unknown”. df - data.frame() for(n in json_data$results){ year -substr(n$date, 0, 4) month - substr(n$date, 5, 6) day - substr(n$date, 7, 8) author - substr(n$byline, 4, 30) title - n$title if(length(author) 1){ author - unknown } Next, we can reassemble the date into a MM/DD/YYYY formatted string and convert it to a date object: datestamp -paste(month, /, day, /, year, sep=) datestamp - as.Date(datestamp,%m/%d/%Y) And finally before we leave the loop, we should add this newly parsed author and date information to a temporary row and add that row to our new data frame. newrow - data.frame(datestamp, author, title, stringsAsFactors=FALSE, check.rows=FALSE) df - rbind(df, newrow) } rownames(df) - df$datestamp Our complete loop should look like the following: df - data.frame() for(n in json_data$results){ year -substr(n$date, 0, 4) month - substr(n$date, 5, 6) day - substr(n$date, 7, 8) author - substr(n$byline, 4, 30) title - n$title
- 26. Chapter 1 ■Background 19 if(length(author) 1){ author - unknown } datestamp -paste(month, /, day, /, year, sep=) datestamp - as.Date(datestamp,%m/%d/%Y) newrow - data.frame(datestamp, author, title, stringsAsFactors=FALSE, check.rows=FALSE) df - rbind(df, newrow) } rownames(df) - df$datestamp Note that our example assumes that the data set returned has unique date values. If you get errors with this, you may need to scrub your returned data set to purge any duplicate rows. Once our data frame is populated, we can start to do some analysis on the data. Let’s start out by pulling just the year from every entry, and quickly making a stem and leaf plot to see the shape of the data. Note ■ ■ John Tukey created the stem and leaf plot in his seminal work, Exploratory Data Analysis. Stem and leaf plots are quick, high-level ways to see the shape of data, much like a histogram. In the stem and leaf plot, we construct the “stem” column on the left and the “leaf” column on the right. The stem consists of the most significant unique elements in a result set. The leaf consists of the remainder of the values associated with each stem. In our stem and leaf plot below, the years are our stem and R shows zeroes for each row associated with a given year. Something else to note is that often alternating sequential rows are combined into a single row, in the interest of having a more concise visualization. First, we will create a new variable to hold the year information: yearlist - as.POSIXlt(df$datestamp)$year+1900 If we inspect this variable, we see that it looks something like this: yearlist [1] 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2012 2011 2011 2011 2011 2011 2011 2011 2011 2011 2011 2011 2011 2011 2011 2011 2011 [30] 2011 2011 2011 2011 2010 2010 2010 2010 2010 2010 2010 2010 2010 2010 2009 2009 2009 2009 2009 2009 2009 2008 2008 2008 2007 2007 2007 2007 2006 [59] 2006 2006 2006 2005 2005 2005 2005 2005 2005 2004 2003 2003 2003 2002 2002 2002 2002 2001 2001 2000 2000 2000 2000 2000 2000 1999 1999 1999 1999 [88] 1999 1999 1998 1998 1998 1997 1997 1996 1996 1995 1995 1995 1993 1993 1993 1993 1992 1991 1991 1991 1990 1990 1990 1990 1989 1989 1989 1988 1988 [117] 1988 1986 1985 1985 1985 1984 1982 1982 1981 That’s great, that’s exactly what we want: a year to represent every article returned. Next let’s create the stem and leaf plot: stem(yearlist) 1980 | 0 1982 | 00 1984 | 0000 1986 | 0 1988 | 000000
- 27. Chapter 1 ■Background 20 1990 | 0000000 1992 | 00000 1994 | 000 1996 | 0000 1998 | 000000000 2000 | 00000000 2002 | 0000000 2004 | 0000000 2006 | 00000000 2008 | 0000000000 2010 | 000000000000000000000000000000 2012 | 0000000000000 Very interesting. We see a gradual build with some dips in the mid-1990s, another gradual build with another dip in the mid-2000s and a strong explosion since 2010 (the stem and leaf plot groups years together in twos). Looking at that, my mind starts to envision a story building about a subject growing in popularity. But what about the authors of these articles? Maybe they are the result of one or two very interested authors that have quite a bit to say on the subject. Let’s explore that idea and take a look at the author data that we parsed out. Let’s look at just the unique authors from our data frame: length(unique(df$author)) [1] 81 We see that there are 81 unique authors or combination of authors for these articles! Just out of curiosity, let’s take a look at the breakdown by author for each article. Let’s quickly create a bar chart to see the overall shape of the data (the bar chart is shown in Figure 1-17): plot(table(df$author), axes=FALSE) Figure 1-17. Bar chart of number of articles by author to quickly visualize
- 28. Chapter 1 ■Background 21 We remove the x and y axes to allow ourselves to focus just on the shape of the data without worrying too much about the granular details. From the shape, we can see a large number of bars with the same value; these are authors who have written a single article. The higher bars are authors who have written multiple articles. Essentially each bar is a unique author, and the height of the bar indicates the number of articles they have written. We can see that although there are roughly five standout contributors, most authors have average one article. Note that we just created several visualizations as part of our analysis. The two steps aren’t mutually exclusive; we often times create quick visualizations to facilitate our own understanding of the data. It’s the intention with which they are created that make them part of the analysis phase. These visualizations are intended to improve our own understanding of the data so that we can accurately tell the story in the data. What we’ve seen in this particular data set tells a story of a subject growing in popularity, demonstrated by the increasing number of articles by a variety of authors. Let’s now prepare it for mass consumption. Note ■ ■ We are not fabricating or inventing this story. Like information archaeologists, we are sifting through the raw data to uncover the story. Visualize Data Once we’ve analyzed the data and understand it (and I mean really understand the data to the point where we are conversant in all the granular details around it), and once we’ve seen the story that the data has within, it is time to share that story. For the current example, we’ve already crafted a stem and leaf plot as well as a bar chart as part of our analysis. However, stem and leaf plots are great for analyzing data, but not so great for messaging out about the findings. It is not immediately obvious what the context of the numbers in a stem and leaf plot represents. And the bar chart we created supported the main thesis of the story instead of communicating that thesis. Since we want to demonstrate the distribution of articles by year, let’s instead use a histogram to tell the story: hist(yearlist) See Figure 1-18 for what this call to the hist() function generates.
- 29. Chapter 1 ■Background 22 This is a good start, but let’s refine this further. Let’s color in the bars, give the chart a meaningful title, and strictly define the range of years. hist(yearlist, breaks=(1981:2012), freq=TRUE, col=#CCCCCC, main=Distribution of Articles about Data Visualizationnby the NY Times, xlab = Year) This produces the histogram that we see in Figure 1-5. Ethics of Data Visualization Remember Figure 1-3 from the beginning of this chapter where we looked at the weighted popularity of the search term “Data Visualization”? By constraining the data to 2006 to 2012, we told a story of a keyword growing in popularity, almost doubling in popularity over a six-year period. But what if we included more data points in our sample and extended our view to include 2004? See Figure 1-19 for this expanded time series chart. 1980 1985 1990 1995 2000 2005 2010 2015 yearlist Histogram of yearlist Frequency 30 25 20 15 10 5 0 Figure 1-18. Histogram of yearlist
- 30. Another Random Documenton Scribd Without Any Related Topics
- 31. first, yet soonbegan to show favorable returns for Lincoln. Betting was heavy in Snagtown. In a few days the leaders began to threaten a dissolution, and, no sooner was it ascertained beyond a doubt that Mr. Lincoln was elected than they proceeded to put their menaces into execution. At this time secession was rife, the very air was full of it. Southern politicians alleged that Mr. Lincoln was a sectional candidate, pledged to the overthrow of slavery. On the 20th of December, 1860, a convention in Charleston declared that the union before existing between South Carolina and other States, under the name of the United States of America, was dissolved. By the 1st of February, 1861, through the influence of the press and the devices of a few leaders, Mississippi, Florida, Alabama, Georgia, Louisiana, and Texas, following the example of South Carolina, had passed ordinances of secession, and their Senators and Representatives left their seats in the American Congress. On the 4th of February, delegates from six of the seceded States met at Montgomery, Alabama, and formed a union under the title of the Confederate States of America. For provisional President they elected Mr. Jefferson Davis, of Mississippi, who had been a Colonel of some note in the Mexican War, a member of Pierce's cabinet, and a prominent advocate of Southern rights in the United States Senate. But we must now attend to the individuals in this history, whom other historians have neglected. On the evening of the 23d of December, 1860, Mr. Tompkins and his family were assembled in the large, cheerful sitting-room. The fire- place was piled with blazing logs, and the light and warmth of the room seemed more pleasant, contrasted with the soughing winds and falling snow without. No thought of the approaching holidays seemed to have entered the minds of any of the group. The brothers were silent and sat apart. The cloud, so small as to be scarcely discernable, was growing larger
- 32. and overshadowing each.It had first been visible on election day, when they parted on the way to the polls. Though no allusion had ever been made to this conversation, their brotherly union had been shaken. They drove, rode, and hunted together as usual, but there was one question they could never approach without disagreeing, and disagreement was apt to produce disagreeable feelings. There was a ring at the bell, and the girl who answered the summons ushered in Uncle Dan, closely followed by Crazy Joe. Good evenin' to ye all, said the old man, as he entered the cozy sitting room. How do you all do? Pretty well, Uncle Dan. How are you and Joe this evening? returned Mr. Tompkins, rising and grasping the hard, rough hand of the old hunter. We ar' both purty well, said Uncle Dan, shaking hands with all present. I tell ye what's a fact, it's gettin' cold out, an' no mistake, snowing just like blazes. Joe, who was in no talkative mood, took a seat in a corner, and fixed his gaze on the fire. I thought from the way the wind whistled it had grown colder. Come, Maggie, fix Uncle Dan and Joe some supper, said the planter. Ya-as, fur I'm hungry as a wolf, returned the old man, with the familiarity of a frequent and welcome guest. Are you hungry, Joe? asked Mrs. Tompkins. I am, but it is written that man shall not live by bread alone, but by every word of God. I'll put that ar' fellur agin any preacher in the settlement for quotin' Scriptur. He jest seems to know the whole thing by heart. Have you heard any news recently? Mr. Tompkins asked.
- 33. News! Don't talkabout news! Jist wait till I've had some supper, an' I'll give ye a little mess o' news that'll make ye hair stand on ye head. After the mountaineer had partaken of a warm meal, and returned to the comfortable sitting-room, Mr. Tompkins asked: What is that remarkable news, Uncle Dan? Wall, I kin tell it now, he answered, resuming his seat, but I sw'ar it war too much for a empty stomach. About two hours ago the news first come to Snagtown, an' now the whole place is wild. The convention, which met at Charleston, South Carliny, three days ago, passed ordernances o' secession, and declar' the State out o' the Union. Oh, pshaw! it must be a mistake, said Mr. Tompkins. Mistake? Not by a jug full. It ar' a actual fact. The news came in as straight as a crow flies. There war rumors o' it before, but now it's sartin. Great heaven! that means civil war. It means war, but it wont be civil, not by a jug full. They ar' already talkin' about musterin' men and gettin' ready to fight. Thar's to be a grand muster and speakin' at Snagtown next Saturday. They say that Mississippi, Florida, Alabama, Georgia, Louisiana, and Texas ar' sure to foller South Carliny, in a few weeks, and maybe all them slave States, even Virginia and Missouri. Have the people gone crazy? cried Mr. Tompkins. It's no more than might be expected, said Oleah. The North has set her foot on the South, and if she feels like withdrawing from the partnership, she certainly has a right to do so. Partnership? put in Abner, with an astonished look.
- 34. It is merelya confederation of States, formed by a compact, and, if one wishes to withdraw, she has the right, answered Oleah. Our Government is formed by the people, and not by the States, said Abner. Then, why is it not called the United People, and not the United States? Each State is a separate corporation, capable of suing and being sued, contracting and dissolving contracts. They were originally colonies, but when they freed themselves from Great Britain, for protection and safety, they united. Who can doubt that South Carolina has not the right, when she has become capable of taking care of herself, to withdraw from others? There is a great difference between corporations and governments, said Abner. Our Constitution does not say, 'We, the United States,' 'As the people of the United States, in order to form a more perfect union.' When they belonged to England, they were considered as a whole and not as a part. In the Declaration of Independence, declaring the Colonies free and independent States, does so in the name and by the authority of the good people whom they represented, and not of the States. All that sounds very well, Abner, said Oleah, bitterly, but words will have no effect on an oppressed and downtrodden people. The South will be free— Yes, if they have to enslave one-half of humanity to do so, interrupted Abner. That's just the point Abolitionists are driving to, though few are as honest as you to admit. The slaves make the South wealthy and powerful. The North is jealous and wants to deprive us of the means of wealth. There is but one remedy left us—the same remedy adopted by the Colonies when oppressed by Great Britain— withdraw, rebel.
- 35. You are toohasty, said Abner, more coolly. You have no assurance that when Abraham Lincoln does take his seat, the 4th of March next, he will abolish slavery. Wait and see. Wait and see? cried Oleah. Wait until he has withdrawn every gun and armed vessel from the South? Wait until he has overrun the whole country with armed soldiers? Wait until he has bound us hand and foot? Then what can we do? No! Now is the time for action. I don't believe Lincoln will free the negroes, said Abner. I will stake my life as the wager, said Oleah, that before his term of office expires, he declares every negro in the United States a free American citizen, war or no war. Mark my words and see if I am not a true prophet. Come, come, boys, we have had political discussion enough for the present, said Mr. Tompkins. Ya-as, said Uncle Dan, we don't want the civil war to commence to-night; least of all places, heah. One thing sure about it, you youngsters had better let us old folks talk 'bout these things, we can do it without gettin' so red in the face. The whole country is in a bad fix, an' ef it comes to a smash up, I swar I don't want to see it begin between brothers.
- 36. CHAPTER VIII. MR. DIGGSIN A NEW FIELD. Mr. Diggs was defeated for the office of county attorney by a large majority, but he was young and buoyant, and after a few days of repining began to revive. A new excitement took possession of him. Strange talk came to his ears, and his little round eyes glistened with delight from behind his glasses, and his little round lips parted with smiles of pleasure. War on a gigantic scale—a new Nation, with new men at its head—was the all-absorbing topic. The Union was shattered, and a new Nation was rising out of the ruins and fragments of the old. Mr. Diggs concluded to espouse the cause of the new Nation. He would raise a company of volunteers to fight its battles; he would be captain. From captain he would be promoted for his bravery to colonel, from colonel to brigadier-general, or commander-in-chief. Mr. Diggs' fertile imagination planned a glorious future for himself. Other men had risen from obscurity to renown, and why not he? He strutted about with his hands thrust deep into his pockets, reveling already in his future greatness. The new and powerful Nation was his all-absorbing theme. When he met any one he would say: Well, what's the news, and what's the prospect of war? The prospect was very good, every one thought. One day, talking with a young man about his own age, but cooler and less blood-thirsty, Mr. Diggs said they were too slow about fighting. Since the surrender of Twiggs in Texas no other event had transpired, and such indifference was monstrous.
- 37. Don't be ina hurry, Diggs, said his friend. Let them have time for consideration. There's no need of consideration. I am ready now. I will go, like Marion, to avenge my country's wrongs, said Diggs. This is war against our own countrymen, said his friend, and I don't think there is any place in either rank for me. There is a place for me, said Diggs, strutting about with his hands in his pockets and expectorating profusely. My country needs me, and I reckon there's a place for me. Will you take a colonelcy to commence with? his friend asked, with a smile. I don't expect a colonelcy at first, said Diggs. I want to start at the foot of the ladder, as captain, and gradually rise until I am commander-in-chief. You would make such a noble-looking general! said a bystander, surveying the fat little fellow. You can talk, Howard Jones, but I—hem! hem!—have always had a taste for military life. You would make such a fine-looking commander, said Jones. Mounted on a tall charger you would yourself strike terror to the enemy. I can prove that all generals were small men, said Diggs, strutting about. Of course they were; but you—you would kill all your enemies. They would die with laughter when they saw a general on a horse seventeen hands high, looking like a bug on a log. Oh, talk sense, Jones.
- 38. On a bigwar-horse you would look very much like a bug on a log, said Jones. But wouldn't it be grand for Crazy Joe's mud man to turn out a general? Can't you talk sense, or are you a fool? roared the exasperated Diggs; and, unable longer to endure the ridicule of his companions, he turned abruptly around and left the crowd gathered about him. The Winter of 1860-61 passed away; but little had been done in Snagtown save mustering and speech-making. Those in favor of open rebellion were in the minority in the neighborhood, but those in favor of neutrality in the majority; but those in favor of standing for the Stars and Stripes the smallest class of all. Patrick Henry Diggs was in a dilemma. His ambition pointed him to the battle-field, that his great abilities, which no one seemed to appreciate, might be shown to the world. The idea of a new Nation dazzled him and showed a path as splendor for his willing feet to follow. But he felt reluctant to draw his sword against the flag of Washington and Marion. He was sure, however, that these turbulent times meant something great for himself. He never lost an opportunity to muster in the ranks of the Home Guards or to make a speech. The eastern part of Virginia seceded on April 17, 1861, but the northwestern portion, about Snagtown, was at peace, save from the mustering of Home Guards to protect home and families from the incursions of either army. Oleah Tompkins was an avowed secessionist, attended the Meetings of the Knights of the Golden Circle, and was already sworn to support the Southern cause. Secret meetings were taking place all over the country, and night meetings held three or four times a week. Mr. Diggs joined one of these secret organizations, and met with them one night in an old school-house which stood on the side of an abandoned road, about four miles from Snagtown in the direction of
- 39. the Twin Mountains.About forty in all had assembled there, among them Howard Jones and Seth Williams, two men who seemed, Mr. Diggs thought, to live only to annoy him. Mr. Diggs had come to the meeting with the intention of making one of his most patriotic speeches; but when he discovered his old enemies, their eyes sparkling with mischief, his heart sank within him. Nearly all present were armed with shot-guns, rifles and pistols, and a guard was placed about the school-house. Preliminary matters settled, Howard Jones rose and addressed the chairman of the meeting, stating that, as they had with them the distinguished attorney, Patrick Henry Diggs, who was in sympathy with the cause, he would like to hear from him. Despite the stirring times, everybody present was eagerly expectant of fun. Cries for Diggs were heard all over the house. Mr. Diggs' opinion of Jones rose rapidly. Mr. Speaker, began Mr. Diggs, rising and gazing about through his glasses, in the language of one of old
- 40. 'I come nothere to talk. You know too well The story of our thralldom.—' Here he made a gesture with both hands, which Jones declared looked like a turtle trying to crawl up hill. 'We are slaves.' A solemn pause. 'The bright sun rises to his course, and lights A race of slaves; he sets, and his last beam Falls on a slave. Friends, Romans, countrymen—' I say, interrupted Seth Williams, in an audible whisper, nudging the orator, s'pose you leave Rome, and come down to our present age. Give us something about the new Confederacy. That's just what I am coming to, said Mr. Diggs, and I hope you will not interrupt me again. After a short pause he resumed: It is no common cause which brings us here to-night. Tyrants and traitors are abroad in the land. A gigantic foe is invading the fair soil of Virginia, and we are here to protect our firesides. All law writers, from Blackstone down, agree that all men should protect their homes. Now, fellow-citizens, remember our forefathers all fought, and bled, and died for this glorious Union. [Applause.] Touch lightly on that, whispered Jones. I repeat said Mr. Diggs, that Washington was the greatest man that ever lived. And now, grown eloquent and excited, he mounted a bench and whipped his left hand under the tails of his coat, while he waved his right in vehement gesture.
- 41. All the effortsof Seth Williams and Howard Jones to keep him on the track were unavailing. He commenced to speak about the Stars and Stripes. Oh, thunder! go back to Rome if you can't make a better secession speech, said Jones. The truth was that Mr. Diggs, like a great many others at this time, hardly knew which side he was on. When he swore to preserve the Union at all hazards, his astonished friends pulled him down. A call was made for volunteers, and Mr. Diggs was the first to enroll his name. Though calling themselves a Home Guard, these volunteers were really enrolled in the army of the Southern Confederacy. Oleah Tompkins was among the first to thus espouse the Southern cause. The clouds of war grew darker and darker every hour. At any moment the storm might burst in all its fury. Snagtown was in a constant state of excitement as the crisis approached. Her more timid citizens trembled with dread. Henry Smith, a farmer's son, a young man of limited education, but of strong common sense, stood in the street one bright morning, engaged in conversation with Seth Williams. Come, now, Harry, said Williams, persuasively, you had better come in with us. The time has come, or will soon come, when our homes will have to be defended. We shall be overrun with soldierly hirelings, who will rob and burn and murder as they go. Our families will need protection, and this duty devolves on us. But, Seth, some say the Home Guard will be marched South into the Confederate army. Oh, nothing of the kind, said Williams. Our only object is to protect our homes from the soldiers of both sides, and to meddle with neither unless they invade our State.
- 42. I think weare justified in protecting our own interests; but, though I despise Abraham Lincoln, I cannot raise my hand against the old Stars and Stripes. Oh, there is no danger that you will be forced into the Confederate army. We are only organizing a Home Guard now; if we raise troops for the South, that will be another thing. When do you meet again? asked Harry. To-morrow night; we go into camp next week in real earnest. Where? On Wolf Creek, about three or four miles away, between here and the Twin Mountains. Where do you meet to-morrow night? At the school-house on the road between here and Twin Mountains. I will be there, said Harry. As Williams walked away, a young man who had been observing the two with keen interest, approached Harry and said: I can tell what you and Seth Williams were talking about. I will give you three guesses, Abner, said Harry, laughing. He was trying to persuade you to enlist in the Home Guards. That was just it, replied Harry. Don't do it, Harry, or you will repent it. I tell you the name Home Guard is only a cover, and every one who enlists will be in the Confederate army in three months. Unless you mean to take up arms against your country, keep clear of the Home Guard.
- 43. I don't wantto fight in Lincoln's army, nor do I want to enter the confederate ranks, so I thought the Home Guards would be the place for me. Don't you enlist, said Abner Tompkins, or you will repent it. As Harry walked away, Mr. Diggs came along, his short legs, in rapid motion, resembling the thick spokes of a wheelbarrow, and his head inclined backward at an angle of forty-five degrees, and his glasses, as usual, on his nose, and his little fat hands thrust deep into his pockets. Hold on, Diggs! said Abner. I want to speak to you. Hem, hem, hem! began Mr. Diggs. Good morning, Mr. Tompkins. Well—hem—I am—that is, I am—hem—glad to see you. I was just going to have my man drive me out to your house. Have a little important business with—that is with one member of your family, he —he he! Diggs, I hear that you have enlisted in the Confederate army; is it so? asked Abner, abruptly. Well, sir, I expect—that is, I apprehend, my dear sir, that—you— perhaps are correctly informed. Why, Diggs, what in the world do you mean? asked Abner. Oh, our country is too large; should be divided. We intend to build up a vast Southern empire. The North has always trampled on our rights, and it is time for us to resist. But how do you intend to resist? By overthrowing the best government the world has ever known? Build up a Southern empire! Is not the grand old republic established by Washington good enough for you? The North is not trampling on your rights. Your wrongs are imaginary. And as to our country being too large, can a nation like ours grow too powerful? Think, Diggs, before you act, or, like Calhoun, you may expect Washington to come to you in sleep,
- 44. and place theblack spot on your hand which Arnold wears in the other world. Think Diggs! Don't raise your hand against your country without well considering the matter. Diggs, for a few minutes, was silent, and then he said: I think you are right, Abner. I will not prove a traitor to my country. I shall ask to have my name taken off the roll to-morrow night. Do so, or you will surely repent it as you live. If you want military honors, seek them in the ranks of your country. There is a call for seventy-five thousand volunteers. You are right, you are right. I will go and volunteer. Where shall I go? We are raising a company at the junction, about twenty miles from here. I will go day after to-morrow, but I am in a hurry now. I am going to your house on business. The fact is—I don't mind telling the facts to you—I am going on purpose to see Miss Irene. He, he, he! I am determined to see how I stand there, he, he, he! Abner started back in amazement, but Mr. Diggs hurried away, without observing his movement. The consummate fool! muttered Abner. The idiot! To think of our Irene! Mr. Diggs hurried off with an air of much importance, and ordered Mose to make ready the carriage, and drive him to the Tompkins mansion. Mose was not as quick of movement as he had been fifty years before, but he managed to have the equipage in readiness by four o'clock in the afternoon. At Mr. Tompkins' door Mr. Diggs alighted, to be informed by Miss Irene's maid that her mistress was calling with Mrs. Tompkins, and
- 45. would not returnfor an hour. I will wait, said Mr. Diggs. I must—hem, hem—must see Miss Irene. After a few moments of waiting Mr. Diggs became tired of sitting in the house and sauntered out to the piazza, and there met the ladies on their return. Miss Irene,—hem, hem, hem, he began advancing. I am delighted to see you, I—hem—that is—hem—I came on purpose to see you, and—and talk with you, and bid you good-by before I leave for the field of glory. I have joined the Confederate army—hem—no, I mean to say I am going to join the Union army in a day or two. That is, I don't know exactly which army I shall join yet—and I come to bid you adieu. Irene looked a little puzzled and felt not a little annoyed at this address. There was something she did not like about Mr. Diggs' manner. Will you come in? she said, and I will see you presently. Mr. Diggs accordingly re-entered the house, and Irene went up to her room to change her dress. She managed to detain herself until tea was announced and then invited Mr. Diggs to the dining-room. After tea the little fellow followed her back to the parlor, and she resigned herself to be bored for an hour or more by him, but did not yet suspect the real cause of his visit. Hem—hem, began Mr. Diggs, Miss Irene, these are troublous times. They are indeed, answered Irene, from her seat opposite the loquacious Mr. Diggs. We don't know one minute what will happen the next.
- 46. No, we donot, said Irene, who really did not imagine what was to happen on this occasion. Hem, hem! two large armies are raising. So I am informed, said Irene. And they mean destruction to each other. I fear some damage will be done. Hem, hem! Sumter has fallen. So I have heard. Deuce take it! thought Mr. Diggs aside, she is as cool as an iceberg, and I am getting flurried. What had I better say or do next? Then a short pause. Some of your friends will doubtless take part in the coming struggle, he finally said. I fear they will be rash enough to do so, she replied. And some may go to return no more,—voice and eyes were growing pathetic. Alas! such is too often the fate of war. I have concluded to enter the army. A great many young men are now talking of going into the army. I feel that my country needs my services. You are patriotic. Mr. Diggs felt flattered. You are—hem—hem, very kind, Miss Irene, to attribute patriotism to me. Patriotism, true patriotism is one of man's most noble attributes.
- 47. I agree withyou. But, Miss Irene, it is hard to go, even to our country's aid, and leave behind friends dearer to us than life. Mercy! mentally ejaculated Irene, does the little fool mean to propose? Then, still without any encouraging warmth in her tone, she asked, When do you expect to leave Snagtown? In two or three days at most, and I feel—hem—pardon me, Miss Irene. He rose and drew his chair nearer hers. He really means it! thought Irene, her eyes bright, half with mischief, half with annoyance. I have something—hem, hem, hem!—I wish to say to you. I—I— that is—hem—I cannot leave for the field of danger until I—have— hem, hem! until I have revealed to you my feelings. Mr. Diggs paused, and tried to look sentimental; but a more sheepish, simple-looking specimen of humanity Irene was sure she had never before beheld. The farce had been carried too far, and she said coldly: Your manner and words are quite incomprehensible, Mr. Diggs. I will make myself plain, said Mr. Diggs, swallowing something in his throat, and taking hope. You shall understand me. I say I cannot leave for the field of battle, cannot face the cannon's mouth, in this suspense— Then don't go, Mr. Diggs, interrupted Irene, with difficulty restraining her merriment, all her pity put to flight by his affectation and conceit. I should almost feel inclined to turn a deaf ear to the 'obstreperous trump of fame,' and 'only list to love and thine,' should you command me to stay.
- 48. Sir, you aregrowing more and more incomprehensible. Let us leave this subject. Not yet, oh no, not yet! Wait until you have heard all. I love you, Irene, dearest, and—and—ah! come to my arms and say you will be mine! Down he went on one knee, with upturned face and out-stretched arms. Poor Irene felt an almost irresistible impulse to laugh, and for a moment dared not speak. He mistook her silence and again began to plead. Speak, O brightest sylph, fairer than the angels, sweeter than— hem, hem!—than the honey in the honey-comb! For mercy's sake, stand up, Mr. Diggs! said Irene. Not until you say you will be mine! and his arms expanded, like an opened double gate. Then Mr. Diggs, I fear you will never reach the field of glory, for the war will be over before you rise from your knees, said Irene. Oh! ah! Hem, hem! You cannot be so cruel,—still kneeling, and leaning further forward, as though to compel her to his embrace. Mr. Diggs, you can never be to me more than a friend. Pray, do not pursue the subject further. Miss Irene, dear, dear Miss Irene, you utterly wreck my life! I care not a straw for it now! whined little Mr. Diggs, turning, still on his knees, towards Irene who had crossed the room, the most pitiful of faces. No answer. You are—hem, hem!—very cruel, Miss Irene, he rose and awkwardly took his seat.
- 49. I regret tohave given you pain, said Irene graciously, as, at Mr. Diggs' request, she rang for his carriage, but I am sure you will soon forget it, and will see that you had mistaken your feelings. As Mr. Diggs was in the act of getting into his carriage the sound of horse's feet came to his ear, and a moment later Oleah Tompkins galloped up to the side of the old rockaway. Halloo, Diggs! are you just leaving? asked Oleah. Yes—hem, hem!—I am going home, said Diggs. Well, be on hand to-morrow night without fail, now. We want every member of the company there, as we shall go into camp in a day or two. Well,—hem, hem, hem!—Oleah, I have almost concluded not to go. I can not—hem, hem!—take up arms against the flag of Washington. Oh, that's abolitionist nonsense! What care you for a flag that will not protect you? That's so, said Diggs. Then why should we consent to bow our necks to tyrant's heels simply because the great and good Washington fought under a rag with certain stripes and certain stars upon it? That is so. Hem, hem, hem! 'They first have breathed treason.' Yes, they stole our property. The interests of the North and South are directly opposite. They want to ruin us, and we must protect ourselves while we can. We can not live in peace with the North; the next best thing is to separate. That's so,—hem, hem!—that's so, said Mr. Diggs. Then why refuse to enter the Confederate army? The South is your country, and if you want military renown seek it in the ranks of your
- 50. country. If theycall you a rebel be proud of the name. Washington and Marion were rebels. Mr. Diggs was completely won back to the Southern cause; and, assuring Oleah he would be with them the next night, drove away.
- 51. CHAPTER IX. THE CHASMOPENS. The storm clouds were gathering dark about the Tompkins mansion. The heads of the household were silent on the question, each knowing the different feelings and sympathies of the other. Their sons were also silent, but there was a sullenness in their silence that foretold the coming strife. There was one member of the once happy household who could not comprehend the trouble, whose very gentleness kept her in ignorance of the threatened danger. Yet neither love nor loving care could keep her from knowing that trouble was brewing. She could not but notice the coldness gradually growing between the two brothers. Brothers whose affection she once thought no earthly power could lessen, were growing daily colder and more and more estranged. Every morning each mounted his horse, and rode away alone, and it was always late in the night when they came home, never together. Gloomy and silent, the morning meal was hurried through, the pleasant conversation that had always accompanied it, was heard no more, if we except the efforts of Irene, who strove with all her power to infuse some of the old-time harmony and brightness into the altered family. It was the evening of Mr. Diggs' visit to the Tompkins mansion, one of those clear bright evenings when the curtains of night seem reluctant to fall, and the fluttering folds seem held apart to reveal the beauty of the dying day. Irene sat by the window, gazing up at the dark blue vault, and listening to the far-off song of a whip-poor- will upon the lonely hillside. Nature to her had never seemed more calm or lovely. The moon, serenely bright, shed mellow light over the landscape, and the dark old forest on whose trees the early buds had swelled into green leaves, lay in a quiet repose. Only man, of all created things seemed unresting. Far down the road she heard the
- 52. clatter of horses'hoofs. At all times now, day and night, she heard them. Clatter, clatter, clatter—sleeping or waking, it was always the same, always this beat of hoofs. To her it seemed as if ten thousand dragoons were constantly galloping, galloping, galloping down the great road: somewhere their marshalled thousands must be gathering. Horsemen singly, horsemen in pairs, horsemen in groups, were galloping, galloping, until her ears ached with the awful din. As she looked, a horseman came dashing down the hill; he passed through the gate and down the avenue. That must be either Abner or Oleah, thought Irene. Six months ago, they would have gone and returned together. When he stepped on the piazza, the moon fell on his face and revealed the features of Abner Tompkins. He came rapidly up the steps and into the house. Staying only a few moments in the room below, where his parents were, then came directly to Irene's door and knocked. She bade him come in. Irene, he said in tremulous tones, I have strange news for you. I must leave to-night for months perhaps, perhaps forever, my home, my parents—and you. Irene sprang to his side eager and excited. Why, Abner, what do you mean? Is it such a surprise to you? I will try to speak calmly, but I have only a few moments to stay. I have a load on my heart that I must unburden to you. What is it? she said, drawing a low stool to his feet and seating herself she took both his hands in her own. Tell me what troubles
- 53. you, let meshare it with you. Who should share your troubles if not your sister? Irene, what I have to say will shock you. No, no, it will not. If you have done anything wrong, I shall be sure it was not your fault— No, you misunderstand me; it is nothing I have done, he interrupted. Then what is this secret, brother? I am not your brother. Irene had promised that his secret should not shock her, yet had a bombshell burst at her feet, she could not have been more astonished. She sprang from the low stool, and stood with clasped hands, the color fading from her face, her slight form swaying as though she had received a blow. Abner, alarmed, sprang from his chair, and caught her in his arms. Irene, Irene, don't take it so, he said, bending tenderly over the white face. Not my brother? Why you must be mad! she gasped. Irene, I am not your brother, but I love you a thousand times more fondly than a brother could love. It was this I wanted to tell you before I leave you. What, Irene, weeping—weeping because I am not your brother! My darling, let me be nearer and dearer than a brother! Abner, I can not realize it, I can not think! she said, pressing her hands to her throbbing temples.
- 54. Think of itwhen I am gone, Irene, for I must go. To-morrow's sun must find me miles from here. But through all the coming strife I shall cherish your image. I shall hope for your love if I return. Now, good-by, my love, my Irene! He caught her in his arms, but it was only a sisterly embrace that Irene returned. She could not yet believe that Abner was not her brother. He went down stairs, she heard his mother's sobs, his father's broken voice; the door opened and closed, and from her window she saw him pass down the avenue, out of sight. Soon she heard a horse galloping down the road, and knew that Abner was riding swiftly away in the gathering darkness. Completely overcome, and not daring to meet Mr. or Mrs. Tompkins till she had controlled herself, Irene, throwing a light shawl about her shoulders, went down stairs, stepped through an open window out on the broad piazza. The cool night air fanned her cheeks and revived her spirits. She walked through the grounds to a summer house covered with trailing vines whose fragrant flowers filled the air with sweetest odors. It can not be, it can not be, she murmured. He was surely jesting. I an outcast or foundling or a oh! merciful Heaven! I can not endure the thought! and her beautiful eyes filled with tears. The whip-poor- will's call still sounded from the distant hillside, and soon another sound broke the evening stillness—the tread of a man's feet on the graveled walk. Irene turned her head quickly, and saw Oleah standing in the doorway. I thought I should find you here, Irene, he said. You always choose this arbor on moonlight evenings. You have been absent all day, Oleah. What fearful business is it that keeps both my brothers from my side!
- 55. Ah! Heaven bepraised, Irene, darling Irene, that you know nothing of it! Abner left to-night, perhaps never to return he said, she went on, wiping the tears from her face. I see you have been weeping, dear Irene. I have more news for you. I too have to bid you what may prove a long farewell. I leave to-night for our camp, and shall soon march to join the main army. But I can not leave you, Irene, without telling you of something I have long kept a secret. Irene could not speak; sobs choked her voice. Then from Oleah's lips fell those same startling words: I am not your brother. She sat motionless. Then it must be true. They could not both be mistaken, could not both possess the same hallucination. If anyone was mad, it was herself. But Oleah went on in his quick passionate way: You are not my sister, dearest Irene, and that you are not gives me only joy. When you were left at our house a tiny baby, I claimed you for my sister, and when I learned you could not be my sister, I said you should one day be my wife. I loved from the first time those bright eyes laughed into mine, and that love has grown with my growth and strengthened with my strength, until it has taken possession of my entire being. O, Irene, Irene, you can never know how deep is the love I have born you from early childhood. I could not leave this old home without telling you that I loved you with more than a brother's love. He paused, and Irene remained silent. Speak, Irene! Will you not speak? She was still silent, her large dark eyes fixed and staring, her white lips motionless, her whole form rigid as a statue. She thought of
- 56. Abner's parting words,and pain and terror filled her soul. Had she entered this happy home only to bring discord, to widen the breach between the two brothers? O Irene, Irene, he pleaded, by the memory of our happy childhood I implore you, speak once more before I go. Say that you will love me, that you will pray for me—pray for my safe return, pray for my soul if I fall in battle! The marble statue found voice. I will pray for you, Oleah, to heaven day and night, for your safe return. But will you give me your love? O Irene, if you only knew how dear you are to me, you will surely learn to love me! I have always given you a sister's warmest love, Oleah, she replied, and this is all too new, too strange, for me to change so suddenly. But you promise you will change? he asked eagerly. I can not promise yet, she said. I do not know myself, and neither do you comprehend your own feelings. Irene, dearest, I have known myself for years. Try to love me, and pray for me, he said, and taking both her hands as she came to his side, for now I must go. He stooped and pressed a kiss on those white lips, and Irene was alone. Soon she heard again the hoof beats of a flying horse, and knew that Oleah had left his home. When he had returned to bid farewell to his home, Abner Tompkins, before entering the house, walked down the long gravel walk, through the avenue of grand old elms, until the outer gate was reached. Here he paused a moment, and gazed up at the moon riding through the dark blue, fathomless vault of heaven; then he turned his gaze upon the spacious pillared mansion, his pleasant home, that he was to leave that night, perhaps forever. It was the
- 57. home of hischildhood; beneath its roof dwelt those he loved; and feelings of sadness filled his heart as he realized the fact that he must leave it. On his right lay the great road, the road that, in his boyhood, he had imagined, led to far-off lands and fairy kingdoms; the road he had thought must be endless, and had desired to follow to its end. Across the road was the forest where he and his brother had so often wandered. Every spot seemed hallowed with sacred remembrances of childhood, and associated with every object and every thought was that brother from whom he was gradually drifting away. He stood beneath the old hickory tree, whose nuts they had gathered, and whose topmost branches they had climbed in their adventurous boyhood. To-night all were fading away. He was going to different scenes, to see strange faces, to meet hardships, danger, perhaps death; worse than all to draw his sword against that very brother whose life had so long been one with his. Oh, what a curse is civil war, said Abner, with a sigh, dividing nations, people and kindred. And, leaning against the trunk of the giant old hickory, he stood for a moment lost in painful reverie. The beat of a horse's hoofs aroused him, and he saw his brother approaching. To reach the house he was compelled to pass within a few feet of the hickory tree, and must inevitably discover Abner, who, however, made no effort to conceal himself. Standing in the shade of the tree as he was, Oleah did not see his brother until he was within a few feet of him, and then could not distinguish his features. Halloo, whom have we here? he said, reining in his horse abruptly. Who is there? Speak quick, or it may be the worse for you, cried impetuous Oleah, not receiving an immediate answer. It is I, Oleah, said Abner, stepping from under the branches of the old tree. The two brothers had grown more and more estranged, but as yet there had been no open rupture between them.
- 58. Well, I mightinquire what you are doing there? said Oleah. And I might ask what you are doing here, and where you are going, and a hundred other questions. If I were to tell you I was star- gazing you would not believe me. I don't know; I might, said Oleah. You were sentimental at times when a boy, and the habit of looking at the moon and stars may have followed you into maturer years. I was just thinking, said Abner, that this tree is very old, yet very hale. It is, answered Oleah; it was a full grown tree when I first remember seeing it. Yes, and we have often climbed its branches or swung beneath them. That is all true, said Oleah, restlessly, but why talk of that, above all other times, to-night? It brings pleasant memories of our happy childhood. And why not to-night as well as any other time? said Abner. I have reasons for not wishing to talk or to think of the past to- night, said Oleah. I have enough to trouble me without bringing up recollections that are now anything but pleasant. Recollections of childhood are always pleasant to me, said Abner, and when storms of passion sway me, such thoughts calm the storm and soothe my turbulent mind once more to peace. Have you been in a rage to-night? asked Oleah, with a smile. No. Then why are you conjuring recollections of the past?
- 59. I have notconjured them up; they come unbidden. This night, above all others, I would not drive the thoughts of our past away. And why? asked Oleah, uneasily. Because this night we part, Oleah, perhaps forever. Oleah, rash, hot-headed, fiery Oleah, had a tender heart in his bosom, and now he was trembling with emotion, although he made an effort to appear calm. How do you know that we are to part to-night? he asked. We are both going from our home, and going in different directions. We are standing on opposite sides of a gulf momentarily growing wider. A fearful suspicion crossed Oleah's mind. Do you leave home to- night? Yes. Where are you going? To join the army of my country and the Union. Oleah started back as if he had received a stunning blow in the face. Abner was aware that Oleah had enlisted in the Confederate army, but Oleah did not dream that his brother would enter the army of the North. Abner, Abner, he cried, hurriedly dismounting from his horse and coming to his brother's side, for heaven's sake say that it is not true! But it is true, said Abner sadly. To-night we separate, you to fight for the cause of the South, I for the preservation of the Union. O Abner, O my brother, how can you be so blinded? It is a war between the North and South, the only object of the North being to
- 60. give freedom toour slaves. You will see if the North should be successful, that every negro in the land will be freed. And you will see that the North has no such intentions. Mr. Lincoln, although a Republican, was born in a slave State, and he will not free the slaves. But, Oleah, it is useless for us to discuss these matters; we part to-night, and let us— But should we meet, said Oleah, his hot blood mounting to his face, it will be as enemies. You are my brother now, but when you don the hated uniform of an Abolition soldier you will be my enemy; for I have sworn by the eternal heavens to cut asunder every tie of friendship or kindred when I find them arrayed against our cause. Oleah, said Abner, be not too rash in your vows. Do not make them just yet. I have already made them; and whoever confronts me with a blue coat and a Yankee musket is an enemy, whatever blood runs in his veins. I pray that we may never meet thus, said Abner. Rather would I have you find among the slain the body of one you no longer own as a brother. One of the stable men now appeared, leading Abner's horse. Oleah's hot passion was gone; his eyes were misty, his voice was choked. The brothers clasped hands in silence, and five minutes later Abner was galloping down the great road.
- 61. CHAPTER X. THE BEGINNINGOF SOLDIER LIFE. A curious scene presented itself at the Junction. But before we attempt to describe the former, we will give the reader some idea of the latter. The Junction was the terminus of one railroad and the junction of two others. One of the railroads led to Washington, one to Pittsburg, and one to Baltimore. It was not a large town; a village of perhaps twelve or fifteen hundred inhabitants, blackened by the smoke of engines. The surrounding country was broken and rough, with hills rising upon hills, deep ravines, rocky gorges, and winding streams, lined with a luxuriant growth of pine and maple, while far away in the distance the gray peaks of mountains could be seen. The Junction was about twenty miles north-east of Snagtown, there being no railroad to the latter place, though there was a hard beaten turnpike, with a daily mail-coach running between the two. Some of the houses about the Junction were of brick, but the majority of wood. There were neat little cottages, looking like fairy abodes, amid the green vines and blooming flowers of Spring-time, and there were cottages neither neat nor fairy-like in aspect; the log hovel, showing signs of decay and neglect. But the village, taken as a whole, was a very pretty place. It was about the 1st of May. The President had called for eighty-two thousand more men, finding seventy-five thousand wholly inadequate to put down the rebellion. Virginia was at this period in a constant state of alarm. Sumter had fallen, Harper's Ferry and Norfolk Navy-yard were in the hands of the rebels, while a mob, in the city of Baltimore, had attacked Massachusetts and Pennsylvania troops on their way to the defense of Washington.
- 62. The Federal Government,on the other hand, was straining every nerve. It had collected about Washington, as speedily as possible, under General Scott, the veteran hero of Chippewa, Lundy's Lane, and the Mexican War, the volunteers who flocked to their country's defense in answer to the President's call. Volunteer companies were raising all over the country. In the extreme Northern States, in the defense of the Federal Government; in the extreme Southern States, in defense of the Confederate Government, and in some of the Middle and Western States, companies were raised for both sides. In fact, there were men in some of the more Northern slave States, who mustered with the rebels and were actually in the Confederate service before they knew it. In Virginia, as we have shown, both sides were represented. The Junction, on account of its railroad facilities, was an important point to guard, and about three hundred volunteers, under Colonel Holdfast, were here stationed. Of these raw recruits, there was but one company that was a complete organization, uniformed and armed at the expense of the Government. It was a company of mounted infantry, under command of Captain Wardle, armed with musket, uniformed in the Government blue, and furnished with horses in order to scout the country. The Government found it impossible to turn out arms and clothing fast enough to supply the volunteers at once, and it was late in the Summer of 1861 before they were all equipped. Many armed themselves, as was the case with two hundred of those at the Junction. Their arms consisted of rifles, shot-guns, and such other weapons as they were able to furnish themselves with. The Junction, as we have said, presented a curious scene. Five tall, white army tents had been erected for Captain Wardle's men, and there were a score or more enclosures, ambitious to be known as tents, made from Virginia wagon-covers, sail-cloth, oil-cloth, sheeting, and bed-ticking. They were of various sizes and shapes; some so small that four men would fill them; others large enough to hold twenty-five. Some of them were square, some round, like
- 63. Indian wigwams, andothers more like a circus canvas than anything we can compare them to. The tents were a motley assemblage, and so, and to a greater extent, were the men therein sheltered. There was first the company of Captain Wardle, properly uniformed and armed, and intensely military in appearance and behavior. They were always drilling when not scouting the country; the raw recruits standing by, overwhelmed with admiration at their easy proficiency in the manual of arms, or the intricate and mysterious movements of the company drill. It was early morning, and the smoke was ascending from half a hundred camp-fires. The scene was a constantly varying panorama of straw hats, linen coats, broadcloth coats, colored, flannel and white shirts. An orderly sergeant was trying to initiate a squad of raw recruits into some of the mysteries of drilling. Remember the position of a soldier, said the orderly. Heels close together, head up, the eyes striking the ground twenty paces away. Now, shoulder arms! Great Moses! Tom Koontz, can't you learn how to handle a gun? Keep the barrel vertical. Do you call that vertical? What d'ye mean by sayin' vartical? asked Koontz. The orderly explained for the hundredth time, that vertical meant straight up and down. He had them then count off by twos, beginning at the right, then he instructed them that at the order of right face, number one was to take a half step obliquely to the right, and number two a step and a half to the left, bringing them in double file at right face. But when he gave the order, half of the men had forgotten their number. Confusion and dismay resulted, and the long suffering orderly sat down and swore until he was exhausted. Camp-life was new to all, and its novelty kept all in a perpetual excitement. There was but little discipline. Officers ordered men and men ordered each other. Every one had suggestions to make, and those who knew the least offered the most of them.
- 64. I tell you,said Sergeant Swords to Corporal Grimm, that tent is not strong. The center pole is too weak, and the guy ropes are rotten. It'll go down. I always knowed them boys didn't know how to fix a tent, said Corporal Grimm, plying his jaws vigorously on a huge piece of pig- tail tobacco. Yes, sir; they've got a good deal to learn yet, said Sergeant Swords, with a sigh. I do hate to see any one, who don't know anything about soldier life, pretend to know so much, said Corporal Grimm, who had had ten days' experience before he enlisted in his present company. So do I, said Sergeant Swords, who had seen at least six days' service. They'll find yet they had better take some one else's advice what's had experience. Why, when I was with Captain Strong's men, and we marched forty miles to Goose Creek Bridge to keep the rebels from burnin' it, we fixed a tent up like that, and the first night after we encamped, there came up a rain-storm, and blowed the thing a quarter of a mile into a brush heap. Did I ever tell you what a hard time we had when I was under General Preston; asked Corporal Grimm, by way of introduction to a story which should redound to his own greatness. No, I believe not, answered Sergeant Swords, with more courtesy than truthfulness, for he had heard the story at least a dozen times. Well, sir, them was tryin' times, said Corporal Grimm, shaking his head and masticating his quid with the air of a man who has suffered. Why, sir, we marched eighty-five miles on foot, and all the rations we got was dried bacon, hams, and crackers. Oh, I just thought I would give anything for something substantial to eat, or a drink of coffee! The boys all run out of tobacco, too, an' we had an awful time. The thought of these hardships brought to his face an expression of extreme agony.
- 65. Why didn't youpress something to eat? You passed through a country where there was plenty, didn't you? asked Sergeant Swords. Yes, but what could fifteen hundred men do at pressin'? Why, they couldn't a got enough to feed one brigade, let alone our whole army, answered Corporal Grimm, who, as much service as he had seen, did not exactly know how many men it took to constitute a brigade. We soldiers have hard times, said Sergeant Swords, brushing some of the mud off his blue jean coat. Wonder how soon we'll draw our clothing and arms? Don't know, but hope soon. I'm tired of these farmer brown breeches. I want a blue coat with stripes on the sleeves. At this moment there came a blast from the bugle. Roll call, said Sergeant Swords. A general gathering of each company about the Captain's tent followed. Abner Tompkins was First Lieutenant of the company of which Sergeant Swords and Corporal Grimm were members. He had been with the company now for over a week. The morning drill was over, and the volunteers were lounging about the tents, on the grass; Abner was leaning with his arm across the saddle-bow of his faithful horse, that he was about to turn out to graze. The mind of the young lieutenant was full of fancies and memories. His sudden departure from home, his interview with Irene, the parting with his brother, all were fresh in his thoughts, and his eyes naturally wandered back toward the road that led to his home. A familiar sight met his view. Coming down the hill, attended by a member of his own company, who had been on picket guard, was his father's carriage driven by the family coachman.
- 66. Welcome to ourwebsite – the ideal destination for book lovers and knowledge seekers. With a mission to inspire endlessly, we offer a vast collection of books, ranging from classic literary works to specialized publications, self-development books, and children's literature. Each book is a new journey of discovery, expanding knowledge and enriching the soul of the reade Our website is not just a platform for buying books, but a bridge connecting readers to the timeless values of culture and wisdom. With an elegant, user-friendly interface and an intelligent search system, we are committed to providing a quick and convenient shopping experience. Additionally, our special promotions and home delivery services ensure that you save time and fully enjoy the joy of reading. Let us accompany you on the journey of exploring knowledge and personal growth! ebookname.com