Reliable and Inexpensive QoS Monitoring in Service Markets

 

We propose a novel infrastructure for Quality of Service (QoS) monitoring that promises a significant reduction in monitoring costs. Our approach leverages an incentive-compatible reputation mechanism in order to accurately estimate QoS by aggregating quality ratings from the clients.

 

Managing change becomes increasingly important in a dynamic, fast-evolving economy. Organizations consequently need flexible IT systems that easily support new business processes and strategic objectives. Service oriented computing enables the construction of such applications, by orchestrating together different services that offer basic functionality. Recent efforts have materialized into standards and tools that significantly facilitate the construction and interoperability of services.

As services can be developed and run by different organizations, they are generally provided under a contract (or Service Level Agreement) that fixes the type and quality of the service to be provided, as well as penalties if these are not met. For example, a payment service that is part of an online shopping site may incur certain penalties if it fails frequently. Another example is providing a communication service where a certain bandwidth is guaranteed and penalties may be incurred if these guarantees are not met. Yet another example is the provisioning of computation or data management services through a network, where a certain efficiency and capacity is guaranteed in an agreement and penalties may be incurred if these guaranteed services are not reached.

An essential requirement for such service provisioning is to be able to monitor the QoS that was actually delivered. As the monetary value of individual services decreases, the cost of providing accurate monitoring takes up an increasing share of the cost of providing the service itself. For example, with current technology, reliably monitoring the quality of a communication service requires constant communication with a neutral third party and would be almost as costly as providing the service itself. The cost of this monitoring remains a major obstacle to wider adoption of a service-oriented economy.

As a joint project between the Artificial Intelligence Lab of Ecole Polytechnique Fédérale de Lausanne (EPFL) and the University of Lugano we are developing alternative mechanisms for monitoring QoS. The basic idea in our approach is to estimate the quality delivered by a service based on the feedback provided by clients. Clients run the monitoring code, and periodically report feedback to a reputation mechanism. The reputation mechanism aggregates the reports and outputs QoS estimates for each service. The advantages of this approach are that (i) the RM gets information about most transactions without actually being a bottleneck (there are no real-time constraints for reporting feedback, and the result of several interactions may be compressed in one feedback message), (ii) the monitoring process is as precise as possible (an immediate consequence of the first point), and (iii) service providers cannot directly tamper with the monitoring process.

 

 
FIGURE 1: Traditional approaches to QoS monitoring involve: (i) centralized monitors that proxy every interaction, (ii) centralized monitors that sample the interactions or (iii) decentralized monitors. In our approach, most reports come from clients. A small percentage of interactions may also be sampled by a specialized monitor. Clients get paid such that it is in their best interest to report honestly.

 

 

A practical mechanisms must, however, ensure that clients report feedback honestly. Consider for example a client who knows that by reporting negatively, the resulting QoS estimate might entitle him to collect penalties from the service provider. The client can tamper with the monitoring code to make the provider pay penalties and obtain a share of these. The novelty of our approach is to use economic incentives rather than hard security to obtain reliable information. Clients get paid for reporting feedback, and the amount depends on the values of other reports submitted by other clients. These feedback payments can be designed such that it is provably best for every client to report the truth. In game-theoretic terms, honest reporting becomes a Nash equilibrium where any individual lie decreases the expected payment of the reporter. Misreporting is thus uninteresting rather than impossible.

Initial experiments provide very encouraging results. We have implemented a QoS monitoring framework based entirely on existing standards and technologies. The extensions required for the collection and payment of feedback are lightweight, and do not introduce significant overhead. The payments for feedback can be made as low as one percent of the cost of the service, and the overall cost of monitoring can be decreased by an order of magnitude as compared to traditional techniques.

In our ongoing research, we are exploring an architecture that integrates reputation mechanisms with service directories and a publish/subscribe infrastructure. Clients subscribe for types of services they are interested in and are notified upon significant QoS changes of certain services or upon the availability of matching services. Clients leverage this infrastructure to continuously evolve and improve their service-oriented applications. We will continue using game-theoretic tools to ensure that the information distributed by the reputation mechanism is reliable, and to ensure that self-interested parties have the rational incentive to adopt the behavior recommended by our mechanisms.

Contact details:

Radu Jurca
Walter Binder
Boi Faltings

Selected papers: