Skip to content

This is a bidding system implemented using Scala with the libraries: akka-actor, akka-http, and akka-streams. Its docker image is created using abt-native-packager and it is tested on minikube.

Notifications You must be signed in to change notification settings

felipegutierrez/bidding-system

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

35 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Build Status Coverage Status CodeFactor Codacy Badge Codacy Badge Lines of code Docker Image Size (latest by date)

Bidding system

Yieldlab is a technology service provider, connecting suppliers (those who have space to show ads, e.g. on their websites) to bidders (those who actually want to show ads). The core process is to listen for requests, gather metadata and bids, and afterwards to determine who is winning. This challenge is about setting up a simplified version of this core process as its own application.

Contents of this document:

1 The task

Build a bidding system behaving as following:

For every incoming request as described in 1, send out bid requests as described in 2 to a configurable number of bidders [5]. Responses from these bidders as described in 3 must be processed. The highest bidder wins, and payload is sent out as described in 4.

Incoming and outgoing communication is to be done over HTTP. Message formats are described below.

Please write code that you would want to maintain in production as well, or document all exceptions to this rule and give reasons as to why you made those exceptions.

Please stay with commonly known frameworks for easier reviewing and explaining afterwards.

1.1 Incoming Requests

The application must listen to incoming HTTP requests on port 8080.

An incoming request is of the following format:

http://localhost:8080/[id]?[key=value,...]

The URL will contain an ID to identify the ad for the auction, and a number of query-parameters.

1.2 Bid Requests

The application must forward incoming bid requests by sending a corresponding HTTP POST request to each of the configured bidders with the body in the following JSON format:

{
	“id”: $id,
	“attributes” : {
		“$key”: “$value”,
		
	}
}

The property attributes must contain all incoming query-parameters. Multi-value parameters need not be supported. Test is after starting the bidders using the scripts/test-setup.sh script, then send some json file:

http POST localhost:[8081|8082|8083] < src/main/resources/bidders-request-[1|2|3|4|5|6|7|8|9|10|11].json

1.3 Bid Response

The bidders' response will contain details of the bid(offered price), with id and bid values in a numeric format:

{
	"id" : $id,
	"bid": bid,
	"content": "the string to deliver as a response"
}

1.4 Auction Response

The response for the auction must be the content property of the winning bid, with some tags that can be mentioned in the content replaced with respective values.

For now, only $price$ must be supported, denoting the final price of the bid.

Example:

Following bid responses:

{
	"id" : 123,
	"bid": 750,
	"content": "a:$price"
}

and

{
	"id" : 123,
	"bid": 500,
	"content": "b:$price"
}

will produce auction response as string: a:750

1.5 Configuration

The application should have means to accept accept a number of configuration parameters. For the scope of this task, only one parameter is to be supported:

Parameter Meaning
bidders a comma-separated list of URLs denoting bidder endpoints

2 How to test your application

In order to test your application for correctness, a simple test suite is provided.

2.1 Prerequisites

First, a set of bidders is required that will respoond to bidding requests sent out by your application. For this test suite, we will be using a pre-built Docker image that will be started several times with sligthly different configuration values.

Moreover, we provide a shell script that executes the tests and verifies the test results. That shell script requires the curl and diff binaries to be in your PATH.

So, here is a list of the requirements:

2.2 Start the Docker containers

To start the test environment, either use the script test-setup.sh or run the following commands one after the other from your shell:

docker run --rm -e server.port=8081 -e biddingTrigger=a -e initial=150 -p 8081:8081 yieldlab/recruiting-test-bidder &
docker run --rm -e server.port=8082 -e biddingTrigger=b -e initial=250 -p 8082:8082 yieldlab/recruiting-test-bidder &
docker run --rm -e server.port=8083 -e biddingTrigger=c -e initial=500 -p 8083:8083 yieldlab/recruiting-test-bidder &

This will set up three bidders on localhost, opening ports 8081, 8082 and 8083.

2.3 Start the application

You can use the following configuration parameters to connect to these bidders from your application:

Parameter Value
bidders http://localhost:8081, http://localhost:8082, http://localhost:8083

2.4 Run the test

To run the test, execute the shell script run-test.sh. The script expects your application to listen on localhost:8080. It will issue a number of bid requests to your application and verify the responses to these requests. If your application doesn't respond correctly, it will print out a diff between the expected and the actual results.

3 Bidding system in action

3.1 Requirements

  • JDK version 1.8+
  • scala version 2.12+
  • sbt version 1.4.7+

3.2 Quick test

After start the bidders using the script test-setup.sh, start the Bidding system by running the following command. Please use ", the argument name --bidders. and separate the bidders by a single , without spaces.

sbt "run --bidders http://localhost:8081,http://localhost:8082,http://localhost:8083"

Run the generic test using the script run-test.sh and make sure that the 3 bidders are running.

3.2.1 Basic commands:

  • Compiling: sbt compile.
  • Compiling & Running: sbt "run --bidders http://localhost:8081,http://localhost:8082,http://localhost:8083".
  • Unit tests: sbt test.
  • Generate documentation: sbt doc, then open the target/scala-2.12/api/index.html file.

3.3 Fault tolerant system

The Bidding system accepts bidders using a fault tolerance approach. In case that some or all bidders passed as argument to the Bidding System are not available, the Bidden System will compute the highest bid based on the bidders that are responding before 5 seconds. If the answer of a bidder last more than 5 seconds it will be considered a null bid and will not be processed. Hence, the bid request will never fail, regardless there are bidders available or not. Test it by killing some bidder(s) already running (i.e., that were passed in the argument list) and issue single HTTP GET command available at the script run-test.sh. The bid offer may change depending on the bidders that are available.

3.4 Docker image

We use the sbt-native-packager to generate the Docker image of the Bidding System.

sbt docker:stage
sbt docker:publishLocal
docker images
 
REPOSITORY                        TAG                     IMAGE ID       CREATED          SIZE
felipeogutierrez/bidding-system   0.1                     5284993293f2   20 seconds ago   127MB

docker run --rm --add-host host.docker.internal:host-gateway -i -p 8080:8080 felipeogutierrez/bidding-system:0.1 --bidders http://host.docker.internal:8081,http://host.docker.internal:8082,http://host.docker.internal:8083

docker image rm -f IMAGE_ID

3.5 Kubernetes

TODO: the kube-file to start the bidders containers using docker are not starting using kubernetes. This is happening because the parameters "server.port=8081 biddingTrigger=a initial=150" are not recognized by the docker image inside Kubernetes. Therefore, all the bidders start on the default mode on port 8081 and we cannot have a winner bid in this environment.

$ minikube start --cpus 4 --memory 8192
$ kubectl create namespace bidding-system
$ kubectl apply -f k8s/bidders-pod.yaml -n bidding-system
$ kubectl apply -f k8s/bidding-auction-system-pod.yaml -n bidding-system
$ kubectl -n bidding-system port-forward bidding-auction-pod 8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
kubectl get services -A
kubectl get pods -A
kubectl -n bidding-system logs -f bidder-01-0
kubectl -n bidding-system logs -f bidder-02-0
kubectl -n bidding-system logs -f bidder-03-0
kubectl -n bidding-system logs -f bidding-auction-pod
kubectl -n bidding-system exec -i -t bidding-auction-pod -- /bin/ash


kubectl -n bidding-system delete pods bidder-01-0
kubectl -n bidding-system delete pods bidder-02-0
kubectl -n bidding-system delete pods bidder-03-0
kubectl -n bidding-system delete pods bidding-auction-pod
kubectl delete namespace bidding-system

About

This is a bidding system implemented using Scala with the libraries: akka-actor, akka-http, and akka-streams. Its docker image is created using abt-native-packager and it is tested on minikube.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published