If you’re interested, you can download it here [PDF, 4MB].  Only 58 pages 1.25 spacing.  Booyah.

The dude even looks like me :(

If you’re interested in participating, take a look at my little recruitment page:

http://grabka.org/internet/flickrsearchtagging/

It’s a multi-step process that starts with you installing some software, and sending me an email with your Flickr username.  After that, it’s all simple :-)

Take a look at the study page if you are interested in participating.  I need about 50 people to do this.

It’s a little utility that enables a couple of things:

1. Let’s you to propose tags for images that don’t belong to you on Flickr. Contributing tags if you want to help describe the image is often not possible, unless: you’re the owner, you’re a contact of the owner, or the person has allowed very permissive tagging rights.
2. Keeps your search queries around the tagging area. Queries are valuable, because you as a user took the time to contribute that text at some point. Now you can leverage that same text when you want to tag an image.
3. Tag your images with the proposed tags, or delete the proposed tags.

If this sounds novel and useful, it is :) Or at least, that’s what I’m trying to prove in my thesis.

Installation

In order to do any of this, you need to:

1. Be using Firefox 1.5+, Flock, and any other browser that can run Greasemonkey 0.6+. This is an untested claim :-)
2. Install Greasemonkey in Firefox, if you don’t have it installed already.
3. Install my magical script: Flickr Search Tagging UserScripts.org

What happens to my contributed tags?

They get stored on a server (atthelib.com), attached to a timestamp, your Flickr account name, and the identifier of the photo you are tagging.

In terms of privacy and control of the tags: there is very little. This whole process is using the “Wikipedia approach”: everyone can contribute. The owner of the image can delete a contributed tag. A deleted, contributed tag cannot be re-added. There are some anti-spam measures, including limits to how many tags can be proposed at one time, in one day, from one location, and so on.

Anywho, this is an alpha release, so any and all feedback is most welcome (feel free to use the comment form below)! This is part of a Master’s thesis, so for now all rights and copyrights are strictly reserved.

Screenshot

In hopes of simplifying things, I’ve settled on a human-centred, conceptual definition of search:

Search refers to the process of a user developing a need, defining a query, retrieving information, viewing result(s), providing feedback, and refinements of those steps.

The end result does not have to be finding a single result. Occasionally, other steps in the search process, such as seeing a result set, can satisfy the users need. For example, if the need was to gain information (“What does a ’87 Oldsmobile Toronado FE3 look like?”) rather than find a specific image (“I need a picture of a black ’84 Cutty!”), viewing the result set may be enough.

Figure 1 is an illustrates the definition of search, in the human and interface context.

Figure 1 – Defining Search – View full size (54KB)

Search begins when a user develops a need for the search, and visits some image search service. The user then enters a query that they believe matches their search need (often is not the case). The form of this query is dramatically affected by the context of the interface: if the interface is just a keyword search bar, the user will enter keywords. If the interface offers a sample-image upload, the user may upload a sample image. If an interface offers domain specific search criteria, it will help the user find relevant images in that domain ex. an automobile search service that allows users to select “modified” vs. “stock” vs. “show” vehicles.

The query is then transformed into something that is useful for the information retrieval system. Information retrieval can include retrieving images as well as their metadata, their categories, or tags or whatnot. Here retrieval refers explicitly to the process done by the computers involved, rather than by the user. The bulk of the challenges that are tackled by researchers involved in CBIR occur around this step (query-to-retrieval-to-result-set).

Once the retrieval process is complete, a result set can be displayed back to the user. The format, order, and other information offered by the interface has a profound impact on a users ability to find a acceptable result, or otherwise navigate the result set. The result set can offer enough information for the user to then want to refine their query, or even refine their initial search need.

While interacting with the result set the user can choose to select an individual result for closer inspection, selection, or whatever they want. The user may offer feedback based on their reaction to either a single result, or the part of the result set they are interacting with. The interface hopefully allows for some sort of feedback mechanism which feeds into the relevance calculations of the image retrieval system. The result set then will update based on the feedback. Examples of this would be finding images “more like this one”, excluding small images, offering “yes, this is perfect!”

A users act of selecting an image out of a result set is probably important enough to provide at least some relevance feedback. This idea has been supported by research that claims that the vast majority of image searching work is done by interacting with the result set, well before a single image is selected.

The processing of feedback may not have an impact on the result set, but may improve search results in the future. An example of this kind of feedback would be tagging an image that the user viewed. ex. a photo of the Toronto skyline is viewed, and the user adds a tag or comment: “CN Tower at night”. Flickr, Facebook, and other image hosting services heavily leverage this process in their search process.

Anyway, that’s the taxonomy I will be working with. Feedback and commentary are most welcome.

CBIR is missing the point

As a side note, much of the technology mentioned in CBIR papers doesn’t give much credence to the human use element (“Why would people use this?”), nor the interface element (“How would a person make use of this, even if it was useful?”). There are many, many theses worth of papers to be written about: picking an element used in CBIR and studying the corresponding human and interface artifacts ex. searching with an image texture in mind (“fuzzy dog”), and studying what people are expecting to find, and how an interface can facilitate that.

Google  Scholar, at least amongst my peer group, doesn’t need an introduction.  The quality of its results,  the listing of authors on the left hand side, and one click access to “Recent articles” makes it my default search.  But I regularly dip into Live Search, ACM Portal, and (less often) CiteSeer to get a more comprehensive list of papers.

Live Search Academic, the offering from Microsoft, has a very neat interface that allows you to quickly preview the abstracts of the papers in your result set by hovering over them with your mouse.   In addition to searching by relevance (default), number of citations, and by author, you can group by an author (very useful), journal (neat), and conference (sounds useful .. isn’t).  The hover-over BibTex citation export is very quick and simple.

ACM Portal is most useful for me on campus, as the articles are downloadable when my IP is in the range that the Library uses.  As far as finding most recent research, Portal beats both Scholar and Live, hands down.

First, every semester I hope/pray to get a Graduate Teaching Assistant job, which luckily gets easier and easier as I accumulate “seniority points.”

Second, no funding means no specified project, which means freedom to choose any research topic I please, as long as my (very lenient/forgiving) advisor is OK with it. Well, it’s been about eight months since I’ve come back from India all ready to start researching, and only two days ago did I actually settle on a topic.

Eight months is a long time to pay tuition, and follow dead ends with literature reviews. Also, those months are expensive if you waste your time on partying, girls, video games, Union involvement, student government, keggers, new housemates, motorcycles, trips to Mississippi, Vancouver, Ottawa, and so on. Well .. maybe it wasn’t a complete waste, per se :)

Finally, I’ve settled on a topic that I’m truly interested in.

Usability and Image Search

The topic: usability of image retrieval interfaces for systems based on image content, rather than user-provided textual meta-data.

What does this mean? Image searching (like Google Image, or Flickr search) is actually really, really complicated stuff. The systems on the back-end of the process use a variety of properties of an image to catalogue it in their vast library: the file name, it’s dimensions, data size, human contributed tags and categories, and a whole slew of other things. That specific information is called the metadata. Most of major (Yahoo, Google, MSN, etc.) image search engines also scour the web-page content that surrounds an image to get more metadata that can be searched.

Human contributed metadata is particularly meaningful: tags, categories, and file names point to the content of the image, not just its properties. The problem with human contributed metadata is that it’s:

• very incomplete
• very subjective, thus possibly inaccurate
• very time consuming if you do want to make it accurate or complete
• there are no standard descriptive elements in markup languages like XHTML or HTML for people to tag their images on the web

If you have a photo of a man riding a bicycle in Greece, unless you’ve tagged that picture with “man”, “bicycle”, “greece” (in four or five of the most common languages in the world), the likelihood of that image being catalogued so it can be found by an interested party is quite low.

Researchers since the mid-1990′s have spent a lot of time and energy in a field that hopes to automate more and more of that descriptive process, so that images can be searched on more than their properties. The field is called Content Based Image Retrieval (CBIR). Rather than relying on metadata to describe images, one can have a computer “see” the image and record information about it: colour, texture, salient regions, shapes, layout, etc. This is the content of the image. Cataloguing all of the useful content of a very large number of images accurately, completely, and in a way that can be easily searched, is the holy grail of this field. We are decades away from such a system, but major strides in that direction have already taken place.

Humans + CBIR

A smaller consideration of all of this is: how is a person supposed to search for something based on content? As in, I know a dog is fuzzy, I know that the computer can find fuzzy textures, but how do I describe to a search engine that I want to find all images of a brown, fuzzy dog? It sounds simple, should be simple, but is not simple in the current state-of-the-art.

Usability is a field that concerns itself with how things are used, especially interfaces of computer programs. I’m very interested in usability of these image search systems; image search systems that already exist, and proposed systems that will exist once we have the technology to automatically identify brown fuzzy dogs in your Flickr photos from the cottage.

So that’s where I’m at. I have a general idea of the area I’m interested in, so I’m in the process of reading some review papers about the state of CBIR. Hopefully, a literature review is coming next.

Abstract

This paper addresses issues of Denial of Service (DoS) for Web Services. Architecture of Web Services is described in context of being doubly susceptible to denials of service (DoS), and contrasted against other models of delivering software. The impact on the client-publisher relationship is discussed, given the impact of loss of functionality during DoS. Solutions involving performance testing and connection security are proposed.

Introduction

The Internet is an ideal candidate as a platform for offering business services. Thus it is the foundation of the recent emergence of the Web Services standards. Despite the success of the underlying structure, Web Services are structured in way that contrasts the architecture of the Internet. Web Services emulate the structure of the enterprises that offer the services, by specializing functionality at the nodes in the network, rather than distributing functionality amongst its clients. This architecture makes clients specifically susceptible to Denials of Service. Enterprises must employ safeguards to ensure that the functionality they are offering as Web Services will not be subject to Denials of Service.

This paper begins with discussing the architecture of the Internet, specifically the distributed nature of its core functionality. Denial of Service is detailed as it relates to the servers that are used as hosts of Web Services. The architecture of Web Services is discussed, and it is argued that preventing Denial of Service is a particularly critical concern as Web Services are doubly sensitive to Denial of Service attacks: at the Internet host, and the Web Services itself. Finally, solutions which work to prevent Denials of Service in Web Services are proposed.

The Internet and Denial of Service

As the host for Web Services, it is important to understand what makes the Internet as robust and available as it is. The variety of services that make up the Internet architecture are designed in such a way that each end point of the Internet can be self-sustaining, based on the requirements of the client and decentralized security considerations[GGKL89]. This functionality includes Internet Protocol (IP) addressing, domain name (DNS) resolution, content serving (HTTP for Web, SMTP for E-Mail, and many services), and content caching. The client’s ability to run a fully functional Internet node is limited by cost and expertise, but this expertise is very much within the reach of a typical Information Technology (IT) department. This allows Internet service providers (ISPs) to provide all of the required Internet services for its clients. This functionality is not only available to ISPs and IT departments, but each client node on the Internet; most Linux distributions are packaged with all of the tools necessary to recreate the Internet network, for example.

The strength of this distributed approach is that if one or many remote nodes in the network fail, the overall functionality is maintained for the other nodes. The content at those remote nodes may be unavailable, but the functionality of the Internet tasks is not affected as a whole. Most importantly, Internet services at the other nodes can continue to function. This is in contrast to many distributed software architectures, including Web Services, which concentrate unique, required functionality in each of the nodes.

Denials of Service

Every node on the Internet that is assigned a unique Class A IP address is accessible from the rest of the Internet. Those exposed nodes often services such as web servers or email servers for other parties to access. As a downside to their general availability, these services can be subject to attacks, including Denial of Service (DoS) attacks.

DoS attacks involve overwhelming a system with requests in a given period of time to the point where the system cannot further respond[MSB+06]. The server is designed to respond to all requests, or at least attempt to minimally process them, even to just to make the decision to discard the request. The sum of processing power required to deal with the flurry of requests overwhelms the system resources, and prevents the system from functioning properly. Adversaries can attack the externally exposed functionality of Internet nodes; the Transmission Control Protocol (TCP) layer of HTTP (web) servers are particularly targeted, and have been exploited by denial of service attacks since the early 2000s [MSB+06].

Requests stemming from a DoS attack can be malformed, incomplete, properly formed but meant to induce resource intensive actions, or formed to specifically take advantage of specific service vulnerability. DoS attacks can come from a single client, or from multiple clients simultaneously[MSB+06]; this is called a Distributed Denial of Service attack (DDoS), and provides new challenges for system administrators who wish to prevent or stop an attack[Cha02]. There is much published research which details mechanisms for stopping and preventing DoS attacks for servers that communicate using TCP. But research and technical documentation which focuses on preventing DoS attacks at the layer of the Web Service is unfortunately limited. The DoS can potentially happen at the SOAP messaging layer, lower at the XML parser, or higher up at the functionality enabled by the Web Service. The W3C identifies DoS attacks using SOAP messages as a threat for message level security in WS-Architecture notes, thus raising awareness of the issue[Dav04].

The W3C identifies other message level security threats, including man-in-the-middle attacks and spoofing[Dav04]. Man-in-the-middle attacks involve intercepting messages between the client and server by a third party. The intercepted messages can then either be monitored or injected with other data. Spoofing involves faking the address or other identifier in a request, in order to emulate a legitimate or trusted client, or to maintain anonymity by falsifying the adversary’s true identity. These attacks are of particular, as they are often used in combination with DoS[MSB+06]; this hinders efforts to block the adversaries using firewalls, or other routing solutions.

Web Services and Denial of Service

The architecture of Web Services often mimics the design of the enterprise, rather than that of its Internet host. An enterprise offers specialised services, and thus has some of those services exposed as Web Services. In a traditional Internet setup, business-critical functionality is often hidden away from the exposed services offered by the node, away from the risk associated with attacks. Business services that are not exposed through the web server are thus not victimized to the same extent as exposed services; these services can continue to function properly during an attack. For example, billing, ordering, and customer relationship management should not be affected by the loss of the sales and product information website, regardless of the close relationship of those components.

With the provision of Web Services critical business processes are exposed over the Internet. Thus any enterprise that delivers Web Services is now responsible for delivering and securing their Internet availability, as it becomes a carrier of their business process. Web Service providers must be aware that the availability of their service is critical to their clients as much as it is to them.

Web Service DoS and Clients

The functionality and availability of Web Services becomes critical to the clients that use it, as well as to the enterprise that hosts the service. The failure of the node that provides the Web Service means immediate failure of that functionality in the rest of the system. This may be completely crippling to clients, and detrimental to the business process of the provider. The extent of the disability will depend on the nature of the service being provided, the frequency of use of that Web Service within the rest of the clients system, and other design considerations. Web Service functionality is critical to the enterprise that publishes it, as it would be rare for an enterprise to take the time and energy to integrate a Web Service into its process if it was not critical to the success of the process.

With the adoption of an enterprise’s Web Services by a client, the software written for or by the client now critically depends on the reliable, secure, and steady provision of the Web Service. This is different from the model of service offered using localized software, where the service is provided locally at the client site by installing the software on the client’s IT resources. Software that is reliant on Web Services is structured in a way that is not distributed or localized; instead the software explicitly depends on the availability of the functionality offered by a remote node in the system. When adversaries attack Web Services, DoS attacks critically affect all participants in the service. Clients are thus more susceptible to Denials of Service with Web Services than with other localized or distributed models of delivering services through software.

Double Sensitivity

Web Services can be targeted with DoS attacks in two ways: attacking the Web Service directly with native messaging protocols like SOAP, or attacking the underlying service providing transport protocol. Web Services rely on transport protocols such as HTTP and SMTP, which are provided by servers which are vulnerable to DoS, man-in-the-middle attacks, and spoofing attacks. Thus Web Services are vulnerable to all of the attacks that are present in the architecture of the Internet, as well as attacks on the Web Service itself. Additionally, Web Service availability is affected by general Internet availability. This availability can be affected by congestion, slow response times due to busy or improperly balanced networks, routing between networks, and other network concerns that are not directly tied to the Web Service.

The Web Service server itself is vulnerable at many layers. DoS attackers can target the often intensive task of parsing XML documents. Clients as well as attackers have access to a complete functional description of the service via exposed WSDL files. Using WSDL information attackers can formulate requests which are known to be particularly resource intensive. WSDL though can no more be considered a vulnerability than providing publicly accessible API documentation. Thus securing the WSDL may be as imperative as securing any internal system documentation.

Web Services are Resource Intensive

It can be argued that the Web Service architecture, much like most modern remote function execution architectures, is quite resource intensive for the host. Microsoft has tested two of their remote function platforms, .Net Remoting and ASP .NET Web Services. Microsoft has found that the response time and the number of requests handled per second with the web service standard SOAP format performs as much as 60% worse than remote calls in a binary format[Pri02]. Not all toolkits and software platforms are created equal; some perform far better under load than others.

Industry has mirrored this concern that Web Services implementations are resource intensive. In late 2006 media outlets reported that given the choice between Amazon.com Web Services and simpler representational state transfer (REST) interface, one in five of development clients choose Web Services[Tim06]. This choice is driven by a variety of factors, but anecdotal evidence collected from Internet blogs indicates that response speed of the web service implementation weighs heavily. Jeff Barr of the Amazon Web Services team has been quoted as saying the performance of the REST functionality is up to six times faster than the equivalent web services functionality at Amazon[Ada03].

Any Internet service can limit DoS vulnerability by maintaining the ability to deal with a large number of requests, be they valid or invalid. Thus selection of a platform to develop Web Services must include performance evaluation in order to handle. Particularly important in testing for DoS resilience is measuring at which point denial of service occurs, and controlling resource use appropriately. DoS is far more critical in Web Services than in other software services since, as explained before, client systems are critically affected.

Protecting Web Services from DoS

In the quest to protect Web Services from DoS, several solutions appear, each with their own set of tradeoffs. The solution can involve establishing dedicated, secured connections between the client and provider of the Web Service. Additionally, providers can ensure service by maintaining very high thresholds for performance, which involves testing the performance of the Web Service software platform. Lastly, service providers must limit the Internet exposure of functionality, especially if it is resource intensive. Detecting these attacks is an area of research receiving some research attention[PSKC06], but avoiding them through design and architecture solutions is the promoted approach.

The W3C notes that in order for a web service to operate properly, it must function securely[Dav04]. This includes securing the message exchange, publication of the Web Service, discovery of the Web Service. Solutions to security problems require many different security mechanisms, including authentication and authorization of identities, encryption of the message traffic, and auditing of policies between the client and the server. In practical terms, prevention of DoS attacks in Web Services can be difficult as measures taken to ensure authentication and authorization may in fact increase the performance burden of the Web Service, and accelerate DoS.

Several solutions exist which can prevent DoS attacks on enterprise services which rely on the conditions of the connection between the client and service. Each solution requires establishing a trusted relationship between the client and publisher of the service. Dedicated connections between parties that rely on each others functionality reduce the risk of DoS, as well as general network congestion affecting service availability. Dedicated connections are not exposed to general Internet traffic which avoids external DoS attacks. DoS attacks can originate from trusted nodes within those dedicated connections, which emphasises the need for auditing mutual security policies.

Secured connections can protect against spoofing during DoS attacks, as the identity of the clients within the secured environment is often known, or at least described in such a way that is traceable and blockable in case of an attack. There are many traffic security options, including dedicated connections which are routed outside of general internet traffic, secured software tunnels, and virtual private networks (VPNs).

Additionally, highly emphasising performance evaluation during Web Service development can aid in protecting against DoS. All web services toolkits are not created equal, performance testing of the proposed architecture must include loads that induce denials of service. Most modern programming languages can inter-operate with simpler, faster methods than Web Services, so the choice of using Web Services must be evaluated from a performance standpoint. The proposed solution must meet the performance needs under heavy load conditions.

When designing the Web Service, the developer must limit the publicly available exposed functionality of the business process. The initial point of contact should require authentication of the client, and the initial steps of the authentication process must be resource un-intensive. For example, a developer should avoid making secured connections to internal mail servers to check account availability. Any function calls to the rest of the Web Service interface should demand an authentication token that is very easily dismissed if invalid. Though, for Web Services that are public and rely on public participation to succeed, extra care must be taken to maximize the efficiency and minimize response time, as authentication may be too much of deterrent for widespread adoption of the service. The choice to authenticate and authorize may impact on privacy issues. Though for Web Services that are inter-enterprise, and generally considered private, extra care must be taken to ensure the availability of the service. Security and availability of the service are paramount, and a complete denial of service should be avoided.

Conclusion

DoS of Web Services is particularly cruel to clients, as well as being detrimental to the business process of the publishers. Thus enterprises must be diligent ensure that the functionality they are offering as Web Services will not be taken down by attacks. The WS architecture is structured in way that contrasts the successful architecture of the Internet, but with safeguards in place to secure and isolate connections between the client and service, taking care to performance test the platforms, and by limiting publicly available functionality Web Services can fight back and maintain high levels of availability.

References

[Ada03] Adam Trachtenberg. PHPWeb Services without SOAP. O’Reilly OnLamp, October 30 2003.

[Cha02] RKC Chang. Defending against flooding-based distributed denial-of-service attacks: a tutorial. Communications Magazine, IEEE, 40(10):42–51, 2002.

[Dav04] David Booth, Hugo Haas, Francis McCabe, Eric Newcomer, Michael Champion, Chris Ferris, David Orchard. Web Services Architecture – W3C Working Group Note, February 2004.

[GGKL89] M. Gasser, A. Goldstein, C. Kaufman, and B. Lampson. The digital distributed system security architecture. In Proc. 12th NISTNCSC National Computer Security Conference, pages 305–319, 1989.

[MSB+06] D. Moore, C. Shannon, D.J. Brown, G.M. Voelker, and S. Savage. Inferring Internet denial-of-service activity. ACM Transactions on Computer Systems (TOCS), 24(2):115–139, 2006.

[Pri02] Priya Dhawan. Performance Comparison: .NET Remoting and ASP.NET Web Services. MSDN Library, September 2002.

[PSKC06] Srinivas Padmanabhuni, Vineet Singh, K M. Senthil Kumar, and Abhishek Chatterjee. Preventing service oriented denial of service (PreSDOS): A proposed approach. In ICWS ’06: Proceedings of the IEEE International Conference on Web Services (ICWS’06), pages 577–584, Washington, DC, USA, 2006. IEEE Computer Society.

[Tim06] Tim Anderson. WS-* vs. The REST. The Register Online, April 29 2006.

AJAX is a new term, but it referes to a collection of technologies that have been complete since around 2004. It’s use has been popularized through the groundbreaking work of Microsoft, Google, and smaller firms such as AdaptivePath, who coined the term. AJAX adds the capability of getting new data from a server once a web page has been generated, in hopes of reducing screen refresh, lowering bandwidth utilization; additionally, developers hope to increase user interactivity of web based applications to the level of a regular desktop application. All of this has come at a cost though, including increased complexity, reduced accessibility, and new security threats.

Literature on AJAX

Little academic literature that focuses on AJAX exists, as most of its components have been the result of previous academic and industry work. Much of the literature that exists is in the form of white papers, recommendations, and industry practitioner guidelines. One of the first such guidelines comes from Microsoft, where they discuss using ASP to do server callbacks after the web page has been generated[1]. Microsoft did not coin the usage of the term AJAX; that was done well by Garrett et al. at Adaptive Path, who wrote the root article that contributed to common understanding of how AJAX applications are to be structured, and how they differ from regular web applications[2]. If you’re interested in determining whether AJAX is in fact a model you should be pursuing in developing your own application, there is no shortage of expert opinion on the pros and cons of the technology [3].

Specifically in terms of security concerns, which are always heightened when developing web-accessible applications, there are a whole new set of things to think about [4]. Good industry examples are abound. A great collection of these examples is Ajaxian[5], a website run by seasoned web developers. The components that make up AJAX are not anything that web developers have not used before. The Wikipedia entry on AJAX programming [6] defines these components, and gives context to their use: JavaScript, XMLHttpRequest Class in JavaSctipt, XML as a data transport, and the Document Object Model (DOM) for manipulation of the static web page. Historical information on how AJAX got to the point it did points to an evolution from DHTML, as well as Microsoft’s work in Remote Scripting [6].

For guidance on implementing AJAX within your own work, Apple’s Developer Connection entries [7] are thorough and explore issues related to browser incompatibility. Though, in order to stay on top of all of the browser related issues, a developer can rely on a variety of frameworks and libraries that are kept up to date with functionality and security updates:

• Dojo. An open-source framework being supported by IBM and Sun.
  http://dojotoolkit.org/
• Atlas. The Microsoft AJAX framework that employs ASP.
  http://ajax.asp.net/Default.aspx
• Google Web Toolkit. Googles official AJAX toolkit (as seen in Google Mail and Google Maps).
  http://code.google.com/webtoolkit/
• Prototype. A decidedly simpler library than any of the above.
  http://prototype.conio.net/

You can download my presentation here: Presentation: Ajax – A new hero for the Web?. Some images used in the presentation are sourced from [2].

References

1. Esposito, D. “Cutting Edge: Script Callbacks in ASP.NET.” MSDN Magazine. (August 2004).
   http://msdn.microsoft.com/msdnmag/issues/04/08/CuttingEdge/
2. Garrett, JJ. “Ajax: A New Approach to Web Applications.” Adaptive Path, LLC. (February 2005).
   http://www.adaptivepath.com/publications/essays/archives/000385.php
3. Downes J, Walker J. “Pros and Cons of using Ajax in a CMS.” CMS Watch. (February 2006).
   http://www.cmswatch.com/Feature/143-Direct-Web-Remoting
4. Twynham, S. “AJAX Security.” IT-Observer. (February 2006).
   http://www.it-observer.com/articles/1062/ajax_security
5. Ajaxian. (October 2006) http://ajaxian.com/
6. Wikipedia contributors, “Ajax (programming),” Wikipedia, The Free Encyclopedia. (November 1, 2006). http://en.wikipedia.org/wiki/Ajax_%28programming%29
7. Apple Computer, Inc. “Dynamic HTML and XML: The XMLHttpRequest Object.” Apple Developer Connection. (June 2005).
   http://developer.apple.com/internet/webcontent/xmlhttpreq.html