Digital Persona Project
Home ||| About ||| Contact ||| Resource base


Intelligent Agents: some conceptual issues

A research paper by Felix Stalder, February 2000


After almost two decades of research (pioneered by Negroponte 1970, Kay 1984) the field of "intelligent software agents" is more heterogeneous than ever. Still, there is no common terminology, not even for the most basic terms. Intelligent agents, intelligent interfaces, adaptive interfaces, knowbots, digital proxy, personal agents, persona manager and network agents are all fuzzily defined and mark somewhat different though overlapping areas.

In the following, I will concentrate on personal agents, their characteristics, the conditions of their usability and the social issues that emerge from these. This paper is intended to serve as a background paper to be complemented with an empirical field study.

Personal Agents
In the most general applicable sense, an agent is "one who is authorized to act for or in the place of another." [1] A personal agent, then, would be one who is authorized to act for another person. In the physical world, there are many examples for such a situation: one might give a neighbour a key to one's house to water the plants while one is absent or an executive might have her phone calls screened by a secretary who decide which are important enough to pass on and which are not. An agent acts on behalf of the person who authorizes her because the represented person is, voluntarily or involuntarily, absent. Ideally, however, an agent does not do anything the represented person would not do if she were present.

A personal software agent can be defined as a computer program to which certain tasks have been delegated. Such a wide definition includes programs such a email filters, or software that automatically initiates backups. The term is at times used for such programs but it is useful to narrow this defintion by adding the following elements (Feldman & Yu 1999):


Since all possible events in the external environment can be predicted and all the knowledge about those events encoded in advance, agents need learning capabilities. How they react to new circumstances can be programmed. What they learn cannot. Two qualities are necessary for adaptiveness While autonomy and adaptiveness are essential criteria for a software program to be called "intelligent agent", there are additional qualities that are often, but not necessarily, associated with intelligent agents: Collaborative behaviour and mobility. Collaborative behaviour builds on the concept of social ability. While we often think of one intelligent agent, in fact, most of today's research concentrates on sets of agents or multi-agent systems (MAS). Each agent is given a discrete task. Sometimes they are parallel, such as when their task finding the same information in different sources. They must work together to establish which agent will carry out each task, and how they will merge the information they collect for presentation to the user. [2] A simple example are programs that query several Internet search engines in parallel and then present the combined results

Mobility is important in the context of network-based agents. This concept refers to the ability of agents to migrate in a self-directed way from one host to another on a network, e.g. the Internet, in order to perform their assigned duties. The duties may include gathering information at each host or balancing workload or traffic on the network. Crawlers [3] of search engines are among the most commonly used mobile agents. Mobility can be considered as an extension to the original concept of autonomy.

When all these qualities are put together, we can define intelligent agents as software programs that have the ability to act autonomously on their user's behalf, learn from experience, and, if necessary, collaborate with other agents to achieve a common goal. Whether or not this amounts to "true intelligence" is of secondary importance given the fact that such systems are commonly called intelligent.

Necessary Conditions for Personal Agents
For personal agents to be employed, several things about the relationship between the task(s), the user and the environment(s) must be assumed: Repetitiveness across time, variations across users, intelligibility of user behaviour, and transfer of authority.

First, the use of the application has to involve a substantial amount of repetitive behaviour (within the actions of one user or among users). If this condition is not met, then the adaptive agent will not be able to learn anything, because there are no regularities to learn in the user's actions. Since agents are to be used in a dynamic environment, the speed with which agents adapt or learn is critical. For agents to be useful they need to be able to adapt significantly faster to emerging behavioural patterns than these patterns change, for example, in the context of changing projects at the work place (Mitchell et al. 1994). If it takes the agent 3 weeks to adapt to monthly changes, then its usability will be severely limited.

Second, this repetitive behaviour needs to be potentially different for different users. If this condition is not met, i.e. the repetitive behaviour demonstrated by different users is the same, then a knowledge-based program could do the same job more effectively than an learning agent (Maes 1994).

Third, consistent user behaviour must indicate consistent user intention. That is what the software agent registers as a repetitive action must have been be done for always the same goal. For a want of a better term, we can call this intelligibility: the ability of the agent to understand what the user does. If the user does the same thing to achieve a variety of goals, or, more precisely, if the agent is not able to distinguish subtle differences in the user's behaviour and classifies it as repetitive behaviour then the agent will not be able to act successfully on the user's behalf. For example, an agent might register that a user took the action of opening an email. However, that one action, opening an email, can have several meanings for the user. For example, not all email which is opened is also read.

Finally, as Mitchell et al. (1994, p.90) write, all agent systems require a "transfer of authority between user and system." Without such a transfer of authority, the agent cannot act autonomously which, after all, is the entire purpose of building and using software agents. This can be problematic, since it may not be entirely clear to what exactely this authority is transferred. To clarify this question, it is helpful to think of a computer program is a cultural artifact, in the sense it is not a "neutral machine" but a one that is filled with various cultural and technical assumptions. As such, an agent not only reacts to user's input but also embodies the perspective, creativity and constraints of its creators. Consequently, we transfer authority to make decisions on our behalf to an entity that is defined in the interplay of users and creators interests, which may not be identical in all respects.

These four conditions Ç repetitiveness of action across time, variations of action across users, machine intelligibility of user behaviour, and transfer of authority from the user to the machine Ç need to be meet in order for an autonomous, adaptive agent to work. These conditions, however, also raise a number of vexing social issues.

Social Issues in the field of Personal Agent
As the term personal agent indicates, there is a close relationship between the software program and an individual user who is represented by it. For the agent to be able to represent the user as correctly as possible, that is, for the agent able to do what the user would do personally in the same situation, the agent must know as much as possible about the user in a given domain. Furthermore, the agent must be able to communicate some of this knowledge to other entities. A shopping agent, for example, may be asked to reveal some information about the user to a site it is querying in order to get the info it is seeking. Given the amount of personal information involved in a personal software agent the potential for privacy invasion is real, and readily acknowledged in the literature (Bradshaw 1997; Nwana 1999).

Privacy
In a recent report co-authored by Information and Privacy Commissioner/Ontario (Hustinx & Cavoukian 1999) two main types of privacy threats are identified. First, threats caused by agents acting on behalf of a user (through the disclosure of the user's personal information). Second, threats caused by foreign agents that act on behalf of others (via traffic flow monitoring, data mining, and even covert attempts to obtain personal information directly from the user's agent). In the context of personal agents, the first of the threats is central.

As an agent collects, processes, learns, stores, and distributes data about its user and the user's activities, the agent will possess a wide variety of information which should not be divulged unless specifically required for a transaction. In the course of its activities, an agent could be required or be forced to divulge information about the user that he or she may not wish to be shared. The most important issue, the Privacy Commissioner points out, is one of openness and transparency. As long as it is clear to the user exactly what information is being requested, what purpose it is needed for, and how it will be used (and stored), the user is in a position to freely make decisions based on informed consent. There are many possible scenarios whereby the agent may release information about the user that, while seemingly innocuous, could be potentially significant and adversely affect the user. An agent's visit to an on-line service to determine the cost of concert tickets could generate a wide variety of clickstream data that could ultimately jeopardize the user's privacy.

For example, the on-line service could log an incoming request from the agent for Bob Smith (bsmith@open.net) looking for tickets to a particular concert on a particular night. In and of itself, this information seems relatively innocent. However, if the agent passes along the user's home address so that he can receive the tickets in the mail, then a more sensitive piece of information has been released into the wider cyber-ether about the user. There are numerous examples of how the release of various types of information, knowingly or unwittingly, results in significant repercussions for the user. As a result, in no time at all Mr. Smith is receiving offers from music companies, book clubs, magazines, and travel firms. While a great deal of this type of data mining already occurs today, the potential for even more significant data collection and exploitation of sensitive personal matters (one's finances, relationships, illnesses, insurance, employment, etc.) could result if this information was in the hands of one's agent.

Control
Privacy is part of a much larger issue which is central to the question the social quality of intelligent agents: control. Openness and transparency, as pointed out, are important ways to ensure that the user stays in control over what the agent does. However, openness and transparency run somewhat counter the project software agents which are intended, among others, to "hide the complexity of difficult tasks" (Maes 1994, p.31). As Jaron Lanier (1995) has pointed out, the totally transparent or controllable agent would be, in effect, simply a filter.

On a deeper level, the control issue also arises between the agent's developer and the agent's user. They might not have the same goals in mind. There are many examples in which the interests of the manufacturer clash with the interests of the user. It would be trivially easy to build a VCR that skips the ads when taping a program similarly to the technology which enables a VCR to adjust its recording time to last-minute changes in the programming schedule (e.g. delays). However, it is not in the interest of the manufacturer to hurt the business of the broadcaster, so this option remains unavailable. For the consumer who buys the VCR as a complete package there is little choice.

In software agents, this issue is likely to be much more subtle, but, given the strategic position of the agent as a gatekeeper between (parts of) the environment and the user, much more important. Agents will, in some way or another, adapt to the user behaviour, thus resemble more and more the user, however, they always stay within the parameters built into the technology by its developers. A negative scenario is easily imaginable. Expanding on the VCR experience we can think of an agent that scans TV programs on the user's behalf and records shows that it assumes the user might be interested in. However, it is much more unlikely that we will see an agent that not only records the show but also deletes all unwanted advertisement, if the user wishes so. Futhermore, the adapation between the user and the agent is a mutual one. Not only learns the agent from the user, but the user learns to see the world through the eyes of the agent. If the agent is used as a "filter" then the user sees only what passes the agent. Since the agent is not a "neutral machine" but an cultural artifact, the contraints of this culture (e.g. the limitation of possibilities programming) will influence deeply the way the world can be seen. Jaron Lanier (1995) writes:
"An agent's model of what you are interested in will be a cartoon model, and you will see a cartoon version of the world through the agent's eyes. It is therefore a self-reinforcing model. This will recreate the lowest-common-denominator approach to content that plagues TV. 'You're interested in Balinese ritual, therefore you're interested in travel, therefore you're interested in the Infobahn Travel Game Show!."
In general terms the question can be posed in the following way: How are the interests of the users and the interests of the developer balanced in those cases where they do not overlap? One way to ensure that both interests are reflected equally -- which is necessary with the users are to develop trust into the agent -- is to develop the technology in an open process, that is, by including user participation through a collaborative design process.

Intelligibility
User participation on the development of the agent, not just in its "customization" or "personalization", is likely to also address another of the key issues surrounding software agents: If they are to act on the user's behalf, it is necessary that they see the world through the user's eyes. Since the agent is some sort of a filter Ç to reduce information overload Ç if it does not see with the eyes of the user, the user will end up seeing the world. Such a shift might be subtle, but nevertheless every consequential (on the social dimensions of filtering, see Shapiro 1999). How can be ensured that the software understands correctly what the user does, rather than simply assigning to a certain behaviour a predefined category which may, or may not, match the user's intention?

One of the great promises of agent technology is that the technology can adapt to the user, rather than the other way round. Adaptiveness and sociability are important aspects in this regard. However, considering that the agent will make decisions on behalf of the user, it is important to assure that the agent does not only know the user's "preferences", and can register the users' behaviour, but also that the agent does so in terms and in categories that match those of the user. Understanding what the user really wants will be a much harder problem than simply providing some initial categories which then can be combined to match at least some characteristics of the user.

One of the difficulties arises from the fact the explicit knowledge that the user has of his or her actions might not be adequate to describe what the users does. Particularly routine actions, those which could be done by the agent, are often done based on experience, rather than conceptual knowledge. This makes it problematic do devise the categories, or the ontology,[4] on which the agent can base its adaptation of the user. In order to better understand the relationship between what the agent can do, and what the user would like to delegate, it will be necessary to employ fine-grained methods of gathering data about user behaviour in real-life settings. Unobtrusive observation and other ethno-methodological methods have been developed explicitly to gather such implicit knowledge. They should be used early on in the design process.

The way the various issues sketched above are addressed will deeply impact on the two issues on which success and failure of any real-life agent implementation depends: competence and trust. As Pattie Maes puts it:
Two main problems have to be solved when building software agents. The first problem is that of competence: how does an agent acquire the knowledge it needs to decide when to help the user, what to help the user with, and how to help the user? The second problem is that of trust: how can we guarantee that the user feels comfortable delegating tasks to an agent? Two previous approaches for building interface agents can be distinguished. Neither one provides a satisfactory solution to these problems. (1994, p.31).


Endnotes:
[1] Merrian Webster Online Dictionary, http://www.m-w.com/home.htm: Search term "agent"

[2]Underlying this approach are ideas of self-organization which are also applied in the neighnbouring fields of robotics and artificial life.

[3] A crawler (aka spider or robot) is a program that automatically explores the World-Wide Web by retrieving a document and recursively retrieving some or all the documents that are referenced in it. The algorithm used to pick which references to follow strongly depends on the program's purpose. Index-building spiders usually retrieve a significant proportion of the references. The other extreme is spiders that try to validate the references in a set of documents; these usually do not retrieve any of the links apart from redirections. (http://foldoc.doc.ic.ac.uk/foldoc/foldoc.cgi?spider)

[4] Ontology, in the context of artificial intelligence (AI) means an explicit formal specification of how to represent the objects, concepts and other entities that are assumed to exist in some area of interest and the relationships that hold among them. For AI systems, what "exists" is that which can be represented. When the knowledge about a domain is represented in a declarative language, the set of objects that can be represented is called the universe of discourse. We can describe the ontology of a program by defining a set of representational terms. Definitions associate the names of entities in the universe of discourse (e.g. classes, relations, functions or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. Formally, an ontology is the statement of a logical theory. [based on http://foldoc.doc.ic.ac.uk/foldoc/foldoc.cgi?query=ontology]



References
    Bradshaw, Jeffrey M. (1997). An Introduction to Software Agents. In Bradshaw, Jeffrey M. (ed.) Software Agents. Cambridge, MA: AAAI Press/MIT Press

    Feldman, Susan; Yu, Edmund (1999). Intelligent Agents: A Primer. Searcher (October) Vol. 7, No. 9

    Hustinx, Peter, Cavoukian, Ann (1999). Intelligent Software Agents: Turning a Privacy Threat into a Privacy Protector. Toronto, On; The Hague: Information and Privacy Commissioner/Ontario; Registratiekamer, The Hague

    Kay, Allen (1984). Computer Software. Scientific American Vol. 251, No. 2 pp. 53-59

    Lanier, Jaron (1995). Agents of Alienation. ACM Interactions Vol. 2, No. 3 pp. 66-72

    Maes, Pattie (1994). Agents that Reduce Work and Information Overload. Communications of the ACM Vol. 27, No. 7 pp. 41-40

    Mitchell, Tom; Caruana, Rich; Freitag, Dayne; McDermott, John; Zabowski, David (1994). Experience with a Learning Personal Assistant. Commununications of the ACM (July) Vol. 37, No. 7 pp. 80-91

    Negroponte, Nicholas (1970). The Architecture Machine; Toward A More Human Environment. Cambridge, MA: MIT Press

    Nwana, Hyacinth S.; Ndumu, Divine T. (1999). A Perspective on Software Agents Research. The Knowledge Engineering Review Vol. 14, No. 2 pp. 1-18 [1.11.99]

    Shapiro, Andrew L. (1999). The Control Revolution: How the Internet is Putting Individuals in Charge and Changing the World. New York: Public Affairs