General ANAC Website

Previous Competitions

Eighth International Automated Negotiating Agents Competition

To be held in conjunction with the International Joint Conference on Artificial Intelligence (IJCAI-2017),
in Melbourne, 19-25th August 2017.

 

News

August 17, 2017: ANAC2017 session program was uploaded: here

July 18, 2017: Qualification Round Results of Repeated Multilateral Negotiation (Genius) League was announced.

June 12, 2017: All Leagues finished accepting new submissions..

June 1, 2017: Deadline for submission for Diplomacy Strategy Game League was extended by June 12th.

May 31, 2017: Deadline for submission for Repeated Multilateral Negotiation (Genius) League was extended by June 12th.

May 23, 2017: Deadline for submission for Human Agent Negotiation League was extended by June 12th.

January 18, 2017: Information for ANAC2017 was updated.

October 20, 2016: Intended participation registration was closed.

October 4, 2016: Intended participation registration was opened.

September 2, 2016: ANAC2017 webpage was launched.

 

 

Overview of ANAC2017

The ANAC competition brings together researchers from the negotiation community and provides a unique benchmark for evaluating practical negotiation strategies in multi-issue domains. The previous competitions have spawned novel research in AI in the field of autonomous agent design which are available to the wider research community.

The goals of the competition are:

  • to encourage the design of practical negotiation agents that can proficiently negotiate against unknown opponents and in a variety of circumstances,
  • to provide a benchmark for objectively evaluating different negotiation strategies,
  • to explore different learning and adaptation strategies and opponent models, and
  • to collect state-of-the-art negotiating agents and negotiation scenarios, and making them available to the wider research community.

From this year, the new leagues are set up, adding to the previous Genius-based competition.

 

1- ANAC Repeated Multilateral Negotiation League

EXTENDED Deadline for submission of ANAC Repeated Multilateral Negotiation League is June 12th, 2017 at 23:59 UTC-12 !

Challenge: What are effective negotiation strategies (e.g. bidding, opponent modeling and accepting) when negotiating repeatedly with agents in a multilateral setting?

Entrants

Entrants to the competition have to develop and submit an autonomous negotiation agent that runs on Genius. Genius is a Java-based negotiation platform in which you can create negotiation domains and preference profiles as well as develop negotiating agents. The platform allows you to simulate negotiation sessions and run tournaments. More details can be found by following this link: http://ii.tudelft.nl/genius/

Performance of the agents will be evaluated in a tournament setting, where each agent is matched with other submitted agents, and each set of agents will negotiate in a number of negotiation scenarios. Negotiations are repeated several times to obtain statistically significant results.

A negotiation scenario consists of a specification of negotiation issues and preferences of all negotiating parties. The preferences of a party are modelled using additive utility functions. In this competition, we will not use domain with discounted factor.

 

Rules of Encounter

Negotiations are multilateral and based on a multi-player version of the alternating-offers protocol. Offers are exchanged in real time with a deadline after 3 minutes. The challenge for an agent is to negotiate with two opponents without any prior knowledge of the preferences and strategies of the opponent. As agents negotiate repeatedly with the same agents they may learn from their previous negotiations with these opponents.

Agents will be disqualified for violating the spirit of fair play. The board of the ANAC2017 competition will be the judge on that. The competition rules allow multiple entries from a single institution, but require each agent to be developed independently.

 

Main updates with respect to ANAC 2016

Learning and Adaptation in Multilateral Negotiations

This year, we allow agents to access data from their past negotiation sessions. The agent can access the following information:

  • The name and order of the agents involved in all their previous negotiations.
  • The utilities of the exchanged offers in any previous negotiation session (according to its own utility space). It is worth mentioning that the agents can access their own utility of a given offer. They cannot see the utilities of the other agents in any previous negotiation sessions .
  • The agreement that is reached by the agents it negotiated with.
Agents may use this information learn about and adapt to domain over time, and to use this information to negotiate better with their opponents. For details, please refer to the Genius manual and the frequently asked questions.
FAQ: http://tinyurl.com/ANAC2017GeniusFAQ

 

Multi-player Protocol

The multi-player protocol is a simple extension of the bilateral alternating offers protocol, called the Stacked Alternating Offers Protocol (SAOP). According to this protocol, all of the participants around the table get a turn per round; turns are taken clockwise around the table. The first party starts the negotiation with an offer that is observed by all others immediately. Whenever an offer is made the next party in line can take the following actions:

  1. Make a counter offer (thus rejecting and overriding the previous offer)
  2. Accept the offer
  3. Walk away (e.g. ending the negotiation without any agreement)

This process is repeated in a turn taking clockwise fashion until reaching an agreement or reaching the deadline. To reach an agreement, all parties should accept the offer. If at the deadline no agreement has been reached, the negotiation fails. More information on the SOAP protocol can found at: Reyhan Aydoğan, David Festen,Koen Hindriks, and C. M. Jonker, “Alternating Offers Protocol for Multilateral Negotiation”, In K. Fujita, Q. Bai, T. Ito, M. Zhang, F. Ren, R. Aydoğan & R. Hadfi (Editors). Modern Approaches to Agent-based Complex Automated Negotiation, Springer Japan, 2017 (To appear)

 

Evaluation

The performance of the agents will be determined by the average individual utilities gained by each agent, and the product of the utilities of the agents as a social welfare metrics separately. That means, we have two categories for the winners: individual utility winners and social welfare winners. The teams of the top performing agents will be notified, and the final results and awards will be announced at IJCAI 2017.

It is expected that winner teams will have a representative attending the conference. Each winner team will have the opportunity to give a brief presentation describing their agent at COREDEMA, Workshop on Conflict Resolution in Decision Making.

The teams of the top 8 performing agents will be notified, and the final results will be announced at the AI conference, in IJCAI 2017.

It is expected that teams that make it through to the finals will have a representative attending the conference. Each team in the final will have the opportunity to give a brief presentation describing their agent.

 

Submission

Each participant is expected to submit a negotiation scenario. That is, each group will submit a negotiation domain description and three conflicting preference profiles represented by means of linear additive utility. Submission package: The participants will submit their agent source code and class files (in a .zip or .jar package) as well as their negotiation scenario (i.e., domain.xml, profile1.xml, profile2.xml, and profile 3.xml). Please send your submission package by 1 June, 2017 reyhan.aydogan[at]ozyegin.edu.tr and anac2017[at]katfuji.lab.tuat.ac.jp.

 

Qualification Round Results

Please see the following link: here

 

Questions and Answers

Feel free to ask your questions! Please see the following link: here

 

 

2- ANAC Diplomacy Strategy Game League

EXTENDED Deadline for submission of ANAC Diplomacy Strategy Game League is June 12th, 2017 at 23:59 UTC-12 !

Entrants

Entrants to the competition have to develop a negotiation algorithm for the game of Diplomacy. Diplomacy is a strategy game for 7 players. Each player has a number of armies and fleet positioned on a map of Europe and the goal is to conquer half of the "Supply Centers". What makes this game very interesting and different from other board games however, is that players need to negotiate with each other in order to play well. Players may form coalitions and make plans together in order to defeat other players.

Every participant in this competition must implement a negotiation algorithm using the BANDANA framework. This negotiation algorithm will then be combined with an existing non-negotiating agent (the D-Brane Strategic Module) to form a complete negotiating Diplomacy player. The BANDANA framework is a Java-based platform specifically designed for the development of negotiation algorithms for Diplomacy.

The interesting aspect of Diplomacy, compared to previous editions of ANAC, is that there is no explicit formula that describes your agent's utility function. The goal of a negotiator is to make deals with its opponents that increase its chances of winning. Since Diplomacy is a complex game over many rounds, your agent will only be able to make an estimation of the value of such a deal using some heuristic approach.

Participants are *NOT* allowed to develop a complete Diplomacy player from scratch. Participants must create their agents by extending the AnacNegotiator class from the BANDANA framework with a negotiation algorithm.

More information about Diplomacy can be found here:

More information about the BANDANA framework, and how to implement your agent for the competition can be found here: http://www.iiia.csic.es/~davedejonge/bandana/.

You are downloadable the newest BANDANA Framework 1.3 from here.

 

Rules of Encounter

Negotiations take place in every 'Spring phase' and every 'Fall phase' of the game. The BANDANA framework offers a limited set of deals the players can propose. This is explained in the BANDANA manual. Negotiations are multilateral. A proposal may involve any number of agents between 2 and 7. A proposal is considered 'confirmed' if all players involved in the proposal have accepted it, unless it is inconsistent with earlier confirmed proposals. Proposals are private to those players involved in the proposal. Thus, if player A makes a proposal to players B and C, then only A, B and C will know about it. Players D,E,F and G will not know about the proposal. Any player may make or accept any proposal whenever it wants, so, unlike the Stacked Alternating Offers Protocol, there is no 'turn-taking'. A neutral 'Notary agent' will record all proposals that are being made and accepted, and will send a confirmation message to all players involved in the proposal once all those players have accepted it. A deal is considered a binding agreement if and only if it has been confirmed by the Notary agent.

At the end of each turn, the the D-Brane Strategic Module will select your player's moves. If your player is involved in any deal that has been confirmed then the strategic module will only choose moves that obey that agreement.

 

Tournament Setup

Participating agents will be put in groups of 7. If the number of participants is not a multiple of 7 then (some of) the groups will be supplemented with 1 or more agents provided by the organization. If there is more than 1 group, the best players of each group will advance to the final round.. In each group (and in the final) the players will play a large number of games. In each game players are randomly assigned to any of the 7 'Great Powers'. Each round of the game will last for 30 seconds. A game ends either if any player conquers 18 Supply Centers (a solo victory), or when the players agree to a draw, or if the game reaches the 'winter 1920' phase, in which case a draw is automatically declared.

In case of a solo victory, the winner receives 12 points, and all other players receive 0 points.
In case of a draw:

  • all players eliminated before the end of the game receive 0 points.
  • If there are 2 survivors, each receives 6 points.
  • if there are 3 survivors, each receives 4 points.
  • if there are 4 survivors, each receives 3 points.
  • if there are 5 or 6 survivors, each receives 2 points.
  • if there are 7 survivors, each receives 1 point.

If 2 or more players in a group end with an equal number of points, the total number of Supply Centers conquered in all games is used as a tiebreaker. If players still rank equal, or if the difference between the players is so small that it can't be considered statistically significant, the organization may decide that more games will be played.

The final results will be announced at an AI conference, IJCAI 2017. The teams of the top performing agents will be notified and it is expected that they will have a representative attending the conference.They will have the opportunity to give a brief presentation describing their agent.

 

Questions and Answers

Feel free to ask your questions! Please see the following link: here

 

 

3- ANAC Human Agent Negotiation League

EXTENDED Deadline for submission of Human Agent Negotiation League is June 12th, 2017 at 23:59 UTC-12 !

Motivation

The Human-Agent Negotiation (HAN) competition is proposed in order to further explore the strategies, nuances, and difficulties in creating realistic and efficient agents whose primary purpose is to negotiate with humans. Previous work on human-agent negotiation has revealed the importance of several features not commonly present in agent-agent negotiation, including retreatable and partial offers, emotion exchange, preference elicitation strategies, favors and ledgers behavior, and myriad other topics. To understand these features and better create agents that use them, this competition is designed to be a showcase for the newest work in the negotiating agent community.

 

Summary

The HAN competition will involve each author or group of authors submitting an agent that will be tested against human subjects in a study run through the University of Southern California. The subject pool will be taken from the standard populace available on Amazon’s Mechanical Turk (MTurk) service, with normal filtration done for participants who are ineligible (see Subject Selection, below).

All agents must be compliant with the IAGO (Interactive Arbitration Guide Online) framework and API, which will allow standardization of the agents and efficient running of subjects on MTurk. Agents will all be run on the same single or set of multi-issue bargaining tasks, examples of which are included below (Domain Examples).

Agents will be allowed to communicate on several channels, including a set of natural language utterances that have been pre-selected and curated by the ANAC committee. Other channels include the exchange of offers through visual cues and natural language, preference statements, and emotional displays.

 

IAGO API

IAGO is a platform developed by Mell and Gratch at the University of Southern California. It is intended to serve as a testbed for Human-Agent negotiation specifically. IAGO is a web-based servlet hosting system that provides data collection and recording services, a human-usable HTML5 UI, and an API for designing human-like agents.

New tutorials have been added for IAGO! Source code and an example agent may also be downloaded from the form at the main IAGO page! In addition, a full documentation of IAGO is available from the download site, which is available at http://people.ict.usc.edu/~mell/IAGO. A brief summary is included here.

All agents are capable of using the API to send Events. Events are interpreted by the UI in preset ways that allow a human user to interpret an agent’s intentions. Human users also generate Events that are passed to the agent developer to interpret as desired. Example Events include:

  • SEND_MESSAGE _ sends a natural language utterance to be displayed on the chat log. Agents may send any language they wish, while human participants are restricted to sending from a preset list of utterances.
  • SEND_OFFER _ sends an encoded offer for the multi-issue bargaining task wherein all items are assigned to either the human player, the agent, or an “undecided” section of the offer table. Also sends a pre-coded, descriptive message when sent from the agent to the human player.
  • SEND_EXPRESSION _ sends an emoticon (either Happy, Angry, Surprised, or Sad) to the chat log, and also briefly shows the corresponding emotion on the visual avatar of the agent.
  • OFFER_IN_PROGESS _ provides information that the other player is currently crafting an offer. Must be explicitly sent by the agent developer to the human player.

All Events may be sent with a delay, to allow chaining of related events (for example, an agent designer could send a message, then wait 2 seconds, then follow-up with an offer and an expression simultaneously). Flood protection will prevent messages from being sent too frequently.

Further detail may be found in the IAGO documentation.

 

Subject Selection and Data Treatment

Competition subject participants will be selected from the MTurk subject pool. Subjects will be adults in the US (18 years or older), and will assert that they are permanent residents of the US (this will be verified with IP address tracking). Restriction to the US will be done to reduce cross-cultural effects. Each agent will be tested against 25 participants. Participants will not be re-used or be matched against more than one agent.

Due to the fact that MTurk participants will be US-restricted and natural language statements are used in the utterance set of the competition, participants will also be asked to affirm that their first language is English.

Subject basic demographic information will be collected, and they may be asked a set of verification questions/attention checks to ensure they comprehend and are engaged in the negotiation. Subjects who fail these questions will be removed from the competition and the resulting data set.

The data set collected by the competition organizers will be released to all agent developers/researchers, as with all submitted source code. Researchers not wishing to release source code should contact the organizers directly.

 

Competition Winners and Evaluation

A set of prizes will be awarded to the winners of the competition in two categories.

The first category will be the High Scoring Agent category. The winner will be determined by the agent that, at the end of the bargaining time, has achieved the highest score.

The second category will be the Agent Likeability Category. The winner will be determined by the agent that, following the conclusion of the negotiation and a subsequent survey, rates highest on user feedback questions.

These questions will include questions like:

  • I would use the system again in the future.
  • I cannot recommend this system to others.
  • I think that I would like to use this system frequently.
  • I liked my negotiation partner.
  • I felt like I could trust my negotiation partner.

We maintain the opportunity to examine other categories for “bonus” prizes.

 

Domain Examples

We present here two example domains. A domain similar to these will be selected as the official challenge to the community.

All challenges this year are multi-issue bargaining tasks, which means both the agent and the human participant will negotiate over the same set of items. Items may have differing values to each side. A “full offer” means that all items are assigned to either the agent or the human participant. A “partial offer” means that some items remain on the table and undecided. No offer is considered binding until both players accept the same full offer.

A negotiation will only end when such a full offer is accepted, or the 10-minute time limit for the negotiation has expired. Human participants will have a warning shown when there is only 1 minute remaining. Agents will have access to the current negotiation time at all points, accurate within approximately 5 seconds. In the case that time expires with no full offer, each player will take points equal to their respective Best Alternative To Negotiated Agreement (BATNA).

Note that the IAGO API allows agent designers to read the natural language descriptions of the issues at runtime (e.g., “Issue1” can be understood to be something like “Lumber” or “Luxury Cars”). However, agents will make use of domain-agnostic calls.

Domain Example 1: This challenge is a simple multi-issue bargaining task over resources between two countries. There will be four distinct resources, with five items in each category. The items will have images and descriptions identifying them as either “Oil”, “Iron”, “Foodstuffs” or “Lumber”. The human player will assign a value of 4 points to each Oil, 3 points to each Iron, 2 points to each Lumber, and 1 point to each Foodstuff. The agent player will assign a value of 4 points to each Foodstuff, 3 points to each Lumber, 2 points to each Iron, and 1 point to each Oil. Each player’s BATNA is equal to the value of a single one their highest item (4, for both the human and the agent).

Domain Example 2: This challenge is a smaller task with greater disparity toward values, and more unknowns for the agent player. The human and the agent take on roles as partners at an estate sale. There are three distinct issues. The first issue is “Luxury Cars”, and there are 6 items in this category. The second issue is “Famous Paintings”, and there are 6 items in this category as well. The final issue is “Mansion”, and there is only 1 item in this category. The agent assigns a value of 5 points to each Luxury Car, 3 points to each Famous Painting, and 8 points to the Mansion issue. The human player assigns 6 points to Luxury Cars, 2 points to Famous Paintings, and 8 points to Mansion. That results in the same total in this “mostly distributive” task. The agent and the human both have a BATNA equal to 20 points.

Note than in both domains, the human’s point values and BATNA will NOT be revealed to the agent designers prior to the competition.

 

Natural Language Utterances

  • It is important that we both are happy with an agreement.
  • I gave a little here; you give a little next time.
  • We should try to split things evenly.
  • We should each get our most valuable item.
  • Accept this or there will be consequences.
  • Your offer sucks.
  • This is the last offer. Take it or leave it.
  • This is the very best offer possible.
  • I can’t go any lower than this.
  • We should try harder to find a deal that benefits us both.
  • There’s hardly any time left to negotiate!

 

Additional Rules

Competition participants will be given a test scenario to practice their agents with. However, to prevent hard-coding preference data into agents, a different set of utilities will be used for the actual competition.

There will be no fewer than 3 distinct issues, and no greater than 5. Each issue will have fewer than 20 items.

Issue utilities will adhere to the following rule:

where k is the total number of issues. Succinctly, this means that the total for each side would be the same if that side got every item.

It is strictly forbidden to use any technique by which an agent stores information between participants. This includes methods by which the agent may learn preferences in one game and then subsequently passes that information (through external server communication or otherwise) back to itself in future games.

All 25 participants are to be treated as fresh instances against which the same agent will be run.

Note: Participation in this competition is done in good spirit and for the furtherance of academic knowledge. Attempts to circumvent the rules described herein or described by the ANAC organizers will not be tolerated.

 

Reference

Mell, J., Gratch, J. (2016) "IAGO: Interactive Arbitration Guide Online", In Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems.

Any updates can be found on the main website for IAGO.

 

 

Organising Committee

  • Dr. Reyhan Aydogan, Ozyegin University & Delft University of Technology
  • Dr. Tim Baarslag, Centrum Wiskunde & Informatica (CWI)
  • Prof. Dr. Katsuhide Fujita, Tokyo University of Agriculture and Technology
  • Prof. Dr. Takayuki Ito, Nagoya Institute of Technology
  • Dr. Dave de Jonge, Western Sydney University
  • Prof. Dr. Catholijn Jonker, Delft University of Technology
  • Johnathan Mell, The University of Southern California

 

 

Sponsors

Makoto lab. Inc.

 

 

Contact

For any questions of ANAC2017, the main contact is Dr. Reyhan Aydogan <reyhan.aydogan[at]ozyegin.edu.tr>