Note : Les descriptions sont présentées dans la langue officielle dans laquelle elles ont été soumises.
CA 02708415 2010-08-16
REFERRED INTERNET TRAFFIC ANALYSIS SYSTEM AND METHOD
FIELD OF THE INVENTION
The present invention relates to a computer implemented method and system for
determining
content in social media, and in particular, to a computer implemented method
and system for de-
termining content published by a commentor, an individual, or a web site, and
determining its
effect on referred web sites.
BACKGROUND OF THE INVENTION
The influence of the social media on the effectiveness of commercial web sites
has become an
increasingly important subject. A major problem facing marketers and public
relation profes-
sionals revolves around the prolific use of social media sites and their
effect in directing traffic to
other web sites, for example E-commerce sites and other corporate sites.
Available web analytics software tools, such as Google Analytics and many
other such systems
are designed to collect real-time site traffic information and can provide
resulting statistics
through graphical user interfaces (GUI) and in formatted reports to the
operator of E-commerce
or other sites instrumented with such web analytics tools.
A brief description of web analytics, provided for example by wikipedia
(http://en.wikipedia.org/wiki/Web_analytics), may be useful to give the reader
an introduction to
the subject matter:
"Web analytics is the measurement, collection, analysis and reporting of
Internet data for pur-
poses of understanding and optimizing web usage.
Web analytics is not just a tool for measuring webs ite traffic but can be
used as a tool for busi-
ness research and market research. Web analytics applications can also help
companies meas-
ure the results of traditional print advertising campaigns. It helps one to
estimate how the traffic
to the website changed after the launch of a new advertising campaign. Web
analytics provides
1
CA 02708415 2010-08-16
data on the number of visitors, page views etc to gauge the popularity of the
sites which will help
to do the market research.
There are two categories of web analytics; off-site and on-site web analytics.
Off-site web analytics refers to web measurement and analysis regardless of
whether you own or
maintain a website. It includes the measurement of a website's potential
audience (opportunity),
share of voice (visibility), and buzz (comments) that is happening on the
Internet as a whole.
On-site web analytics measure a visitor's journey once on your website. This
includes its drivers
and conversions; for example, which landing pages encourage people to make a
purchase. On-
site web analytics measures the performance of your website in a commercial
context. This data
is typically compared against key performance indicators for performance, and
used to improve
a web site or marketing campaign's audience response.
Historically, web analytics has referred to on-site visitor measurement.
However in recent years
this has blurred, mainly because vendors are producing tools that span both
categories."
A web analytics tool alone can be an important marketing research tool, but
while it can provide
detailed and summarized information and analysis of the traffic that arrives
at a web site, includ-
ing categorizing sources of the traffic, it cannot determine the reasons for
the traffic arriving.
Accordingly, there is a need in the industry for the development of
alternative and improved
methods and systems, which would take into account individual drivers for
Internet traffic on
commercial web sites.
SUMMARY OF THE INVENTION
Therefore there is an object of the invention to provide an improved referred
Internet traffic
analysis system and method, which would avoid or mitigate disadvantages of the
prior art.
According to one aspect of the invention, there is provided a method of
compiling marketing in-
formation for a client, comprising: (a) crawling blog posts; (b) extracting an
insight related to a
topic from the crawled blog posts; (c) collecting one or more measurements of
traffic to the cli-
ent website, the traffic being referred to the client website by referring
blog posts, the referring
2
CA 02708415 2010-08-16
blog posts including the crawled blog posts; and (d) associating the
measurements of traffic with
the extracted insight.
Preferably, the step (c) comprises collecting the traffic measurements using a
web analytics tool,
for example, collecting one or more events recorded in the client website. In
the embodiments of
the invention, one or more of the following events are recorded: visits to the
client website; a
number of orders of products from the client website; a number of orders of
products from the
client website divided into a category; and a number of downloads from the
client website.
The method further comprises aggregating the measurements of traffic
associated with the in-
sight across the crawled blog posts. One way of aggregating comprises
multiplying each meas-
urement of traffic by a weighting factor associated with respective referring
blog posts to obtain
a product, and summing the products.
In the method described above, the step (a) includes crawling blog posts,
which contain at least
one link to the client website, and the step (c) comprises collecting
measurements of traffic re-
ferred from a subset of the crawled blog posts. Conveniently, the collecting
comprises collecting
measurements of traffic referred from the subset of crawled blog posts, which
are most influen-
tial blog posts.
In one modification to the method, the step (c) comprises collecting
measurements of traffic re-
ferred from at least one additional blog post of the referring blog posts, the
at least one additional
blog post has not been crawled, the method further including crawling the at
least one additional
blog post and extracting the insight from the at least one additional blog
post.
In the embodiments of the invention, the method further comprises:
(e) determining frequently reoccurring insights, which reoccur frequently
enough to appear
to be a driver for the measurements of traffic; and
(0 presenting a plurality of K of said frequently reoccurring insights and
associated meas-
urement of traffic to the client.
3
CA 02708415 2010-08-16
For example, the step (e) may comprise determining the highest number of
insights associated
with one or more selected measurements of traffic, and K may be between 5 and
20.
According to another aspect of the invention, there is provided a computer
readable medium hay-
ing computer readable instructions stored thereon for execution by a
processor, to perform steps
of a method for compiling valuable marketing information for a client,
comprising:
(a) crawling blog posts;
(b) extracting an insight related to a topic from the crawled blog posts;
(c) collecting one or more measurements of traffic to the client website, the
traffic being
referred to the client website by referring blog posts, the referring blog
posts including the
crawled blog posts;
(d) associating the measurements of traffic with the extracted insight.
According to yet another aspect of the invention, there is provided a referred
traffic analysis sys-
tern for compiling marketing information for a client, the system comprising a
processor and a
computer readable medium comprising a computer readable instruction stored
thereon for execu-
tion by the processor, forming:
(a) a crawler module crawling blog posts;
(b) a collector module for extracting one or more insights related to a topic
from the
crawled blog posts;
(c) a web analytics module, collecting one or more measurements of traffic to
the client
website, the traffic being referred to the client website by referring blog
posts, the referring blog
posts including the crawled blog posts; and
(d) an associator module, associating the measurements of traffic with the
extracted one or
more insights.
In the system described above, the associator module comprises an aggregator
module, aggregat-
ing the measurements of traffic associated with the insight across the crawled
blog posts.
The aggregator module comprises means for multiplying each measurement of
traffic by a
weighting factor associated with respective referring blog posts to obtain a
product, and sum-
4
CA 02708415 2010-08-16
ming the products.
The system further comprises a presenter module, determining frequently
reoccurring insights,
which reoccur frequently enough to appear to be a driver for the measurements
of traffic; and
presenting a plurality of K of said frequently reoccurring insights and
associated measurement of
traffic to the client.
The system further comprises an insights database, having computer readable
instructions stored
thereon, for storing the extracted insights for each crawled blog.
The collector module includes an insights dictionary, storing a list of
predetermined insights
terms for determining which insights to be extracted from each crawled blog
post.
The system further comprises a referred traffic dynamics database, storing the
measurements of
traffic associated with the extracted one or more insights.
In the system described above, the presenter module comprises a graphic user
interface module
for presenting the measurements of traffic associated with the extracted one
or more insights.
Thus, a computer implemented method and system for determining content
published by a corn-
mentor, an individual, or a web site, and determining its effect on referred
web sites, have been
provided.
BRIEF DESCRIPTION OF THE DRAWINGS
Embodiments of the invention will now be described, by way of example, with
reference to the
accompanying drawings in which:
Figure 1 illustrates a basic system architecture 100 of the prior art,
comprising any number of
Referrer Sites 102, a Client Installation 104, a number N of Users 106 (User 1
to User N), and a
Web Analytics module 108;
5
CA 02708415 2010-08-16
Figure 2 shows a system architecture 200 according to embodiments of the
invention, including
a Referred Traffic Analysis System 202;
Figure 3 shows an exemplary use-case diagram 300 illustrating a series of
events in the opera-
tion of the improved system architecture 200 of Fig. 2;
Figure 4 is an example correlation overview diagram 400 to illustrate the
operation of the Re-
ferred Traffic Analysis System 202 of Fig. 2; and
Figure 5 shows a flow chart 500 summarizing steps performed in the improved
system architec-
ture 200 of Fig. 2.
DESCRIPTION OF THE EMBODIMENTS OF THE INVENTION
In summary, embodiments of the present invention take available information
from a web ana-
lytics tool that identifies HTTP referrer traffic, and associate or correlate
other events such as
visits, sales, downloads, etc. recorded on a commercial web site, with text
elements found at the
referrer site. These text elements (expressions) comprising from one to a few
contiguous words,
are words or phrases describing, for example, an attribute of a product for
sale at the client site, a
comment on quality or otherwise of the product, or other meaningful terms. In
the following, the
commercial web site will be called a client site, and the text elements will
be called "insights".
Figure 1 illustrates a basic system architecture 100 of the prior art,
comprising any number of
Referrer Sites 102, a Client Installation 104, a number N of Users 106 (User 1
to User N), and a
Web Analytics module 108. The Client Installation 104 includes a Client
Website 110 hosted on
a web server and a User Interface 112. The referrer sites 102 may be social
media sites, such as a
blog, or a news or publication sites carrying reviews for example.
The Web Analytics module 108 may be provided as an in-house tool within the
Client Installa-
tion 104, but it may also be provided from a server of an external
organization such as Google
6
CA 02708415 2010-08-16
(see http://www.google.com/analytics) or WebTrends (see
http://www.webtrends.com) which
serves multiple clients.
For the purpose of more clearly showing the communication relationships,
connectivity over vir-
tual communications links may be described as follows:
the Users 106 browse to the referrer sites 102 over primary Internet
connections 114;
the referrer sites 102 may include a Universal Resource Identifier (URI) which
is effec-
tively a pointer 116 to the Client Website 110;
having acquired a URI, a User 106 (User N for example) may establish a
secondary Inter-
net connection 118 with the Client Website 110, i.e. the user is redirected to
the client website;
as the secondary Internet connection 118 is opened a referrer identifier (REF)
is transmit-
ted to the Client Website 110 and stored there, REF being effectively a
pointer 120 to the referrer
site 102;
the Client Website 110 may communicate with the Web Analytics module 108 over
a per-
manent or periodically established traffic monitoring link 122; and
the User Interface 112 may communicate over a permanent or periodically
established ana-
lytics connection 124 with the Web Analytics module 108.
In a simple sequence of events, a user, e.g. the User N browses to the
Referrer Sites 102, which
may be a social media site, such as a blog, or a news or publication site
carrying reviews, as indi-
cated earlier.
The user may click on a screen element (e.g. a button, or simply text), which
is associated with
the URI pointing to the Client Website 110. As a result, the secondary
Internet connection 118 is
activated, and the user interacts now with the Client Website 110.
Activity on the Client Website 110 is monitored over the traffic monitoring
link 122 by the Web
Analytics module 108 and recorded. Data recorded in the Web Analytics module
108 may then
be processed, for example summarized and formatted, and delivered to the User
Interface 112
over the analytics connection 124, either as a periodic report or
interactively allowing the client
to interactively view the data.
7
CA 02708415 2010-08-16
What processing in the Web Analytics module 108 of the prior art system 100
may also include,
is some listing of the Referrer Sites 102 or their types from which the user
may have been redi-
rected.
What the embodiments of the present invention add to this scenario, is to
provide the client with
additional valuable information by extrapolating from the actual content of
the Referrer Site 102,
combined with the activity recorded by the Web Analytics module 108, possible
motivations of
the user, including ranking different Referrer Sites 102 as to their value to
in bringing business to
the client for example, as well as ranking the "insights" found on the
Referrer Sites 102 as to
their effect on the users' reactions.
General description of the embodiments of the invention
A client has a website (ClientWebsite) and a web analytics tool. Web analytics
tools of various
capabilities are commercially available from companies such as Omniture,
Meteorsolutions,
WebTrends, and others. The web analytics tool runs on, or is connected to, the
same server as the
client's website.
Activity on the client's website is constantly monitored by the web analytics
tool, which collects
statistics of: hits, button presses, enquiries, purchases, etc. as well as the
referrer URL of the site
that is accessing the client's website. The collected results of the monitored
activity are available
to the client in various formats. The collected results may also be
periodically transmitted in a
selected format to Radian6 for further processing.
The client is a client of Radian6 Technologies Inc., to be also referred to as
Radian6 for brevity,
the assignee for this application, who provides a service to the client.
Briefly, a service provided
by Radian6 Technologies Inc, the implementation of which is disclosed in the
embodiments of
the present invention, includes performing additional analysis, specifically
by associating or cor-
relating the information gathered by the web analytics tool of the client with
3rd party informa-
tion gathered by Radian6 Technologies Inc. from publicly accessible 3rd party
sites, e.g. social
media sites on the web, including social networking sites, blogs, reviews,
news agencies, and
8
CA 02708415 2016-10-24
other similar sites which may contain content relative to the client. The
primary key upon
which correlation to Radian6 data is performed, is via the referring URL
field. However, it
is understood that other ways of correlating the collected information are
also possible.
Radian6 maintains a database of the aforementioned 3rd party information,
which is
gathered by a web crawler application periodically and speculatively, for use
in providing
service to clients. This information may already be available and may have
been used in
other types of service provided by Radian6, and described in earlier patent
applications of
the same assignee US 2009/0048904 filed on July 16, 2008; US 2009/0157668
filed on
Dec 11,2009; and US 2009/0281851 filed on May 07, 2009 and issued as US
9245252.
Some 3rd party information may also be included in the web crawler collection
activity on
demand, as a result of new keywords (trigger or search terms) being requested
by the
client.
Not all referrer sites are of interest to Radian6, for example search sites
(Google etc).
Conversely, not all referrer URLs may be known to Radian6, e.g. in-house
sites, or
referrer sites that were active only a short time, and were missed by the web
crawler.
The client has to manually set up the data integration of their web analytics
tool with
Radian6 through a registration process. A typical example of service setup
requires these
fields:
= Client Account Name (and Client ID)
= Name of the Topic Profile
= Application "id" of the report being transferred from the web service
running the
web analytics tool, to Radian6
= API URL that Radian6 will use to request the report
= Supply a username / password if required to request the report from the
web service
= The time of day that Radian6 should request the report from the web
analytics tool
web service (including the time zone)
9
CA 02708415 2010-08-16
= Names of the measures to be included in the report (up to 10 events) from
the web ana-
lyrics report
= These will also become the names used as the menu options in a sample
screen-
shot
= Provide the list of measures in the desired sort order for how the
measures will be
sorted in the menus in the Radian6 user interface
= Note that Radian6 needs to be notified if the measures in the report
change in or-
der to keep the data load running properly
= Domain of the site that measures are being gathered for (i.e
www.radian6.com)
= Multiple domains can be supported
The setup results in defining a database schema, including up to K (K =10)
measures of interest
to the client. There can be any number of sets of 10 measures configured in a
Radian6 web ana-
lytics configuration. It is also understood that K=10 has been selected for
convenience, and gen-
erally K may be higher or lower than 10 as required.
Once a client is set up, their collected set of measurements is periodically
sent in a report to Ra-
dian6 and stored in a rolling database that keeps the current and last N
reports, for possible trends
analysis.
Each time a report is received from the web analytics tool of the client, an
SQL (mySQL) pro-
gram (a data processing job) is run on the collected measurements in
combination with the "in-
sights" collected from the referring sites. The outcome of the data processing
job for the client's
domain comprises one or more basic correlation records, typically of the form:
key word term, provider URL, client website action (e.g. conversion), count
thus tying social media topics and conversations to specific actions and
objectives on the client's
website.
10
CA 02708415 2010-08-16
Conversions are defined by the client, depending on the desires of the client.
If the client desires
to attract a visitor to the client's website and get the visitor to download a
white paper, then that
is a conversion. If the client desires to get someone to buy a widget, then
that is what they define
as a conversion.
The basic correlation records are then further processed by Radian6 to
generate various summa-
ries, which are displayed in an interactive format, at the client's or any
other location.
Detailed description of the embodiments of the invention
Figure 2 shows an improved system architecture 200 according to the
embodiments of the in-
vention, including the basic system architecture 100. But instead of (or in
addition to) communi-
cating the results of the Web Analytics module 108 directly over the analytics
connection 124 to
the Client Installation 104, the improved system architecture 200 comprises a
Referred Traffic
Analysis System 202, which receives formatted analytics data from the Web
Analytics module
108 over a web analytics output connection 204, associates or correlates the
received formatted
analytics data with content gathered from the referrer sites 102 by a Web
Crawler (a crawler
module) 206, and provides the correlation results as "referred traffic
dynamics" over an interac-
tive Internet link 208 to the client, i.e. the User Interface 112.
The Web Crawler 206 may be an available computer program or a service which,
given a prim-
ing source (initial target) and search criteria from the Referred Traffic
Analysis System 202, will
gather many crawled web pages and deliver their content to the Referred
Traffic Analysis Sys-
tem 202 for further processing.
The Referred Traffic Analysis System 202 comprises one or more computers
having processors,
computer storage subsystems, having computer readable media such a memory,
DVD, CD-ROM
or else, network interfaces, and software designed to be executed on said
computers, the software
having computer executable instructions and data stored on the computer
readable media for
execution by a processor, forming various modules of the Referred Traffic
Analysis System 202
as shown in Figure 2.
11
CA 02708415 2010-08-16
The Referred Traffic Analysis System 202 may simultaneously provide an insight
correlation
service to numerous client installations similar to the Client Installation
104, as well as provide
other computational services outside of the scope of the present invention.
The Referred Traffic
Analysis System 202 may reside in a single location, or it may be a
distributed system with com-
puter and storage subsystems located in several locations. In the interest of
clarity, only a simpli-
fied view of the Referred Traffic Analysis System 202, serving a single client
(the Client Instal-
lation 104) is described in the following. Persons conversant with designing
server systems and
server applications may readily conceive of expanded systems which are within
the scope of the
present invention.
The Referred Traffic Analysis System 202 comprises: a Collector module 210, an
Insights data-
base 212, an Associator module 214 including an Aggregator module 215, an
Analytics database
216, a Referred Traffic Dynamics (RTF) database 218, and a Presenter module
220 including a
Graphical User Interface module 221. Each of these modules comprises a
computer readable
program code and/or data stored in a computer readable medium for execution by
a processor.
The Collector module 210 is a software program, stored in a memory, and
running in one of the
computers of the Referred Traffic Analysis System 202, which crawls the
Internet to find Refer-
rer Sites 102, that is social networking as well as other sites that include a
URI pointing to the
Client Website 110. The Collector module 210 includes an Insights Dictionary
222 of "insights",
that is to say, words or short phrases that may indicate for example a value
statement, or gener-
ally terms that are commonly used in social networking sites to denote product
approval. The
Insights Dictionary 222 may also be customized to include terms specific to
the Client Installa-
tion 104 such as product names. As mentioned above, the Insight Dictionary 222
comprises
computer readable program data stored in a computer readable medium for
execution by a proc-
essor.
The contents of each Referrer Site 102 are compared with the insights listed
in the Insights Dic-
tionary 222, and insights found on each Referrer Site 102 are stored in the
Insights database 212
with the Universal Resource Locator (URL) of the respective Referrer Site 102.
12
CA 02708415 2010-08-16
The Analytics database 216 periodically receives selected analytics data from
the Web Analytics
module 108 over the web analytics output connection 204. The selected
analytics data stored in
the Analytics database 216 include measurements, i.e. frequency, time,
referring link, etc., of one
or more of the following events: visits to the client website, number of
orders of products from
the client website, number of orders of products from the client website
divided into a category,
and number of downloads from the client website.
The Associator module 214 is a software program, stored in a memory, and
running in one of the
computers of the Referred Traffic Analysis System 202, which combines data
from the Insights
database 212 and the Analytics database 216 to periodically generate or update
a set of Referred
Traffic Dynamics for storage in the RTF database 218. The Aggregator module
215 of the As-
sociator module 214 aggregates may be used to aggregate the results of the
Referred Traffic Dy-
namics according to predefined criteria, for example
The Presenter module 220 is a software program, stored in a memory, and
running in one of the
computers of the Referred Traffic Analysis System 202, which makes the
contents RTF database
218 accessible over the interactive Internet link 208, preferably through the
Graphical User Inter-
face module (GUI) 221, to the User Interface 112 in the Client Installation
104. The Presenter
module 220 may for example present the RTF data in a website and the
interactive Internet link
208 is a hypertext transport protocol (HTTP) link.
The Collector module 210, the Associator module 214, and the Presenter module
220, may con-
veniently be implemented as Hypertext Preprocessor (PHP) programs, while the
Insights data-
base 212, the Analytics database 216, and the RTF database 218 may be
relational databases
which permits many of the database functions such as sorting, filtering,
associating, etc., of the
software modules (210, 214, 220) to be performed using the Structured Query
Language (SQL).
Figure 3 shows an exemplary use-case diagram 300 illustrating a series of
events in the opera-
tion of the improved system architecture 200, illustrating the interaction
between a User -A
(302), a Blog "A" 304 being an instance of a Referrer Site 102 of Fig. 2 with
the URL value
13
CA 02708415 2010-08-16
"URL A", a Product Website 306 which is an instance of the Client Website 110
of Fig. 2, a
User-i (308), the Web Analytics module 108, and the Referred Traffic Analysis
System 202 of
Fig. 2. The URL
An arrow labeled "writing" from the User -A (302) to the Blog "A" 304
indicates that the User-A
302 at some time Ti writes this social media blog, which includes the written
text as well as a
Link (URI) to the Product Website 306.
An arrow labeled "crawl" from the Blog "A" 304 to the Referred Traffic
Analysis System 202
indicates that the Web Crawler 206 delivers the URL (=URL_A) and the content
of the Blog "A"
304 to the Referred Traffic Analysis System 202 where "insights" contained in
the Blog "A" 304
are extracted and stored, with the key "URL_A", in the Insights database 212.
An arrow labeled "reading" from the User-i (308) to the Blog "A" 304 indicates
that the User-i
(308) is reading the block at some later time T2.
An arrow labeled "user click" from the Blog "A" 304 to the Product Website 306
indicates that
the User-i (308) has clicked on the Link (URI) in the Blog "A" 304, and thus
will be accessing
and browsing the Product Website 306 which includes a Product Description and,
for example, a
"buy now" button which will cause the product described on the Product Website
306 to be
bought by the User-i (308).
A set of arrows labeled 122 represent the traffic monitoring link 122 from the
Product Website
306 to the Web Analytics module 108 which collects relevant events on the
Product Website
306, specifically the URL=URL_A of the Blog "A" 304 through which the Product
Website 306
was accessed by the User-i (308), and the conversion event, that is the fact
that the User-i (308)
has bought the product.
The tracked events on the Product Website 306 due to the visit of User-i (308)
are combined by
the Web Analytics module 108 into a Referred Traffic listing 310 of tracked
events from other
14
CA 02708415 2010-08-16
user visits, and sent at a time T3 over the analytics output connection 204 to
the Referred Traffic
Analysis System 202 where it is stored in the Analytics database 216 (see Fig.
2).
The Referred Traffic report may for example include a statement of the form
[new visitor, came
from URL_A, bought the product], which may be coded in a standard format such
as the
JavaScript Object Notation (JSON).
If the Blog "A" 304 had not already been crawled before, it is now crawled and
its insights and
URL are added to the Insights database 212.
The insights Insight 1 to Insight M of the Blog "A" 304 are then correlated in
the Referred Traf-
fic Analysis System 202 with relevant events listed in the Referred Traffic
listing 310 from the
Web Analytics module 108, including the fact that the User-i (308) who is
reading the Blog "A"
304 has bought the product.
In a similar way, insights of other blogs, for example, are correlated with
the referred traffic at-
tributed to the respective other blogs to generate a Referred Traffic Dynamics
data set 312 stored
in the RTD database 218. The Referred Traffic Dynamics data set 312 is
configured to compute
numeric correlations (totals) between specific insights such as "awesome
product" and actions
such as "bought product". A high relative correlation between a specific
insight (awesome prod-
uct) and the action (bought product) may lead to the plausible assumption that
the insight has
contributed to the user's decision to buy the product.
Figure 4 is an example correlation overview diagram 400 to illustrate the
operation of the Re-
ferred Traffic Analysis System 202, showing: a number of blog readers 402; a
number of blogs
404 labelled "A" to "G", each blog including a number of insights; a set of
referred traffic events
406 relating to each blog when the Client Website 110 is accessed through the
respective blog;
an Individual Referred Traffic Dynamics table 408; and a Referred Traffic
Dynamics Summary
table 412.
15
CA 02708415 2010-08-16
The blog readers 402 are grouped according to which blog they are reading.
Readers which are
merely visitors are shown as white circles, readers which ultimately purchase
a product are
shown as black circles.
The blogs 404, collected by the Web Crawler 206 (Fig. 2), are shown with some
insights they
contain. The text of each blog may contain many more insights, not shown here
because of space
constraints. In any case, the insights in each blog were identified by the
Collector module 210
(Fig. 2), which has matched the text of each blog against terms in the
Insights Dictionary 222.
These insights, along with the URL of each blog containing them, are stored in
the Insights data-
base 212.
The set of referred traffic events 406 is an example of the Referred Traffic
listing 310 (Fig. 3)
that was obtained by the Web Analytics module 108 which tracked the traffic
activity at the Cli-
ent Website 110. It was transmitted to the Referred Traffic Analysis System
202 and stored in
the Analytics database 216 there. Each record of the set of referred traffic
events 406 (referred to
as events records) includes the URL of the referrer site, i.e. the blog, and a
list of events at the
Client Website 110 that pertained to, or originated from, the referrer site.
For example, the first
event record lists 5 visitors and 3 sales against the URL_A.
The Individual Referred Traffic Dynamics table 408 is computed by combining
information from
the blog information 404 in the Insights database 212 with the 406 set of
referred traffic events
406 in the Analytics database 216 based on the blog URL as the key. The
Individual Referred
Traffic Dynamics table 408 is divided into rows and columns, the rows being
indexed with a
primary key based on the name or URL of a blog, e.g. "A", "B", etc., and a
secondary key based
on the insights, e.g. "awesome product", "nice packaging", etc. in the present
example. The col-
umns are indexed by the primary and secondary keys, and by the type of
referred traffic event
410, such as "Visits" and "Sales". It is noted that "Visitors" (in the set of
referred traffic events
406) and "Visits" (in the Individual Referred Traffic Dynamics table 408) are
used interchangea-
bly here, any distinction between repeat visits by the same visitor may or may
not be determined
by the Web Analytics module 108.
16
CA 02708415 2010-08-16
Only two types of referred traffic event 410 are shown in Fig. 4, but there
may be more types of
referred traffic events 410 recorded by the Web Analytics module 108 and the
number of traffic
event columns in the Individual Referred Traffic Dynamics table 408 would
correspond.
From the Individual Referred Traffic Dynamics table 408 it is then possible
for example, that the
blog "A" included 2 insights ("awesome product" and "great taste") and
produced 5 visits and 3
sales. Similarly, the blog "C" included only one insight ("awesome product")
and produced 5 vis-
its and 5 sales.
The data from which the Individual Referred Traffic Dynamics table 408 is
generated exists al-
ready, partially in the Analytics database 216 and partially in the Insights
database 212. Thus, the
Individual Referred Traffic Dynamics table 408 may be generated by the
Associator module 214
at any time, for example periodically at the times when a report needs to be
sent to the client.
For the purpose of recording a time history, and for the convenience in
subsequent processing, it
is convenient to store this table in the RTD database 218 (Fig. 2) however.
The Referred Traffic Dynamics Summary table 412 is computed from the
Individual Referred
Traffic Dynamics table 408 by omitting the blog identifier and simply
aggregating the results
according to some a predefined scheme, for example by summing the entries of
each types of
referred traffic events column against the insights. In the present example,
the result is: 31 visits
and 12 sales for "awesome product", 5 visits and 3 sales against "great
taste", and 9 visits and 2
sales against "nice packaging". Many conclusions may be drawn from such a
result, but one
might conclude for example, that "awesome product" is the most important
insight since it led to
the most visits as well as sales, with 39% of visits leading to sales. On the
other hand, the insight
"nice packaging" does not appear to be as powerful an insight since it only
led to 2 sales in 9 vis-
its (22%). But even though the insight "great taste" led to the fewest visits
(5) it led to the highest
proportion of sales (3, that is 60%). It is also possible to weight the
traffic measurement results
from different blogs by a weighting factor associated with respective
referring blog posts to ob-
tain a product before summing the products. Weighting factors may have been
assigned to blogs
according to a perceived influence of the blog, specific to the client, for
example blogs of prod-
uct review sites may be assigned a higher weighting factor than blogs from
financial news sites.
17
CA 02708415 2010-08-16
The content of the RTD database 218 could be made directly available to the
client, but prefera-
bly the Presenter module 220 is used to provide access to only the client's
data, formatting the
data for interactive access over the interactive Internet link 208,
preprocessing, and thus reducing
the amount of data that needs to be transmitted, and most importantly,
providing the summarized
data as described in the Referred Traffic Dynamics Summary table 412 and
derivations of it such
as trends etc.
In an alternative embodiment of the invention, the Insights database 212, the
Analytics database
216, and the RTD database 218 are combined into a single database stored in a
computer read-
able medium.
Figure 5 shows a flow chart 500 summarizing steps performed in the improved
system architec-
ture 200 of Fig. 2, including steps:
502: Crawl Blog Post "A";
504: Store URL of Blog Post "A";
506: Extract Insights from "A";
508: Retrieve Measurements for traffic from "A";
510: Associate measurements with insights from "A";
512: Aggregate measurements around unique insights; and
514: Present top insights to the client.
The step 502 (Crawl Blog Post "A") is initiated by the Collector module 210
and carried out by
the Web Crawler 206. Blog Post "A" (Blog "A" 304 of Fig. 3) was found by the
Web Crawler
206 on the Internet, because it has a reference (a button) pointing to the
Client Website 110. In
general, all blog posts, as well as other social media sites having a
reference to the Client Web-
site 110, are referred to as "referring blog posts", and may be crawled. In
particular however,
only a subset of the crawled posts that are determined to be influential blog
posts, may be evalu-
ated in the Collector module 210. Certain methods for determining influential
blog posts have
been described in the earlier patent applications of the same assignee cited
above.
18
CA 02708415 2010-08-16
In the step 504 (Store URL of Blog Post "A"), the URL of the Blog Post "A" is
stored by the
Collector module 210 in the Insights database 212.
In the step 506 (Extract Insights from "A"), insights are extracted by the
Collector module 210
from the content of the Blog Post "A" and stored in the Insights database 212.
Extraction is fa-
cilitated by the Insights Dictionary 222 of predefined insight tokens. While
the insight tokens of
the present embodiment of the invention are text elements such as word and
phrases, other types
of insight tokens, for example emoticons, images, audio files, or other
expressions used to indi-
cate an insight on a social media site, are also within the scope of the
invention.
In the step 508 (Retrieve Measurements for traffic from "A"), all web
analytics measurements
are retrieved from the Web Analytics module 108 for traffic to the Client
Website 110 that was
referred from the Blog Post "A", or generally any traffic that was referred
from any of the rele-
vant crawled blog posts, i.e. blog posts that referred traffic to the Client
Website 110. The re-
trieved web analytics measurements are stored in the Analytics database 216
for use in the fol-
lowing step 510.
In the step 510 (Associate measurements with insights from "A"), the retrieved
web analytics
measurements are associated with the extracted insights from the Blog Post
"A", or generally
with the extracted insights from all of the relevant blog posts and stored in
the Individual Re-
ferred Traffic Dynamics table 408 as described in detail above.
In the step 512 (Aggregate measurements around unique insights), the
measurements around the
unique insights identified and associated in the previous steps, are
aggregated into and the aggre-
gated relations stored in the Referred Traffic Dynamics Summary table 412. The
number of rele-
vant blog posts may be on the order of thousands.
In the step 514 (Present top insights to the client), the top K (K = 10, for
example) insights that
reoccur frequently enough to appear to be a driver for the measurements
observed, are presented
to the client. In practice, this means that a certain amount of manual
intervention or iteration may
be required to determine which insights and which measurements are valuable to
the client. The
19
CA 02708415 2010-08-16
number K of top insights may vary, and may be conveniently selected to be in a
range between 5
and 20. It is understood that other values of K are also possible.
The entire process described in the flow chart 500 may be repeated for each
client periodically,
for example daily or weekly. If retrieved web analytics measurements (in the
step 508) are found
to contain a URL of a referring blog post, that had not been crawled and is
not yet in the Ana-
lytics database 216, a special crawl (Step 502) for that missing URL may be
initiated, in order to
extract the insights of that additional blog post (Steps 504 and 506) before
the measurements
may be associated (Step 510).
Thus, new methods and systems have been provided for improving the ability to
exploit the role
of social networking sites in Internet traffic on commercial web sites, and in
e-commerce in gen-
eral.
Any and all software modules described in the present application comprise a
computer readable
code stored in a computer readable storage medium, for example memory, CD-ROM,
DVD or
the like, to be executed by a processor.
Although embodiments of the invention have been described in detail, it will
be apparent to one
skilled in the art that variations and modifications to the embodiments may be
made within the
following claims.
