Tender evaluation and linearity

As two of the founding members of Market Dojo are engineers and the other a physicist, we take our maths very seriously.  One of the reasons for developing our auction platform for the public sector during our first grant is that we wanted to demonstrate how easy it was to set up and execute a weighted tender.  The specific area we want to address here is the scoring system.

We have been baffled by the illogical and non linear nature of many of the scoring systems in the market, and were determined to provide something better.

Firstly we would like to describe how we have approached the problem and then look at why some of the other common techniques on the market are challenging to use from the buyer and supplier side and how some are even fundamentally flawed.

Here is a section of our capability; the full version can be seen here.

We think any scoring mechanism should be linear and straightforward.  Thus if you are running a weighted tender you need to be able to clearly communicate the mechanism for converting the various elements to scores and how they are combined.

[You can try out all that you read here by just registering with Market Dojo for free and experimenting with setting up weighted events in the Sandpit.]

Questionnaire scoring:
We have made the set up for a questionnaire very easy, providing many different question types.  On top of this you can create a questionnaire which is scored and also weighted by question, section or both.  The simplest way to show you what this means is an example output.

Here you can see that this questionnaire is scored and also weighted by question and section and the total weighted score is the final result for the questionnaire for this participant.

Price Scoring:
We have two techniques for price scoring on our application.

Pro Rata: The system scores the price based on where it sits between the lowest price and the qualification price. This is a simple linear method of determining differences between prices. The qualification price is set before the event and is the highest bid you are willing to receive. The lowest price is the current lowest bid in the event. [One could look at a further extension to this method by setting the lowest bound but this presents its own challenges].

You might use this method when you are looking at many complex factors and need to have greater control about making the scoring methodology suit your tender. You can judge your price scoring against your highest acceptable price.

Percentage of leading bid: The is a simple and powerful technique.  As an example, if the highest bid in a reverse auction is £100 and the next best bid is £110, then the lowest bid gets 100% and the next highest bid gets 90%, as it is 10% over.  This is also a linear mechanism which does not need a qualification price.  One potential disadvantage is that if a bid is double the lowest bid, the participant will get a score of zero, although that may be a fair outcome.

One benefit of this method is that the non-price elements can be equated directly to price.  For example if the non-price element is worth about 1% of the estimated final price then you can weight the non-price elements to be 1% and the price 99% and the evaluations works out consistently (see the example below).

This method can lead to difficulties in the public sector arising from rogue bids and/ or participants having the same score of zero when they have different bids.  In the private sector though you have more flexibility and can decline participant during an auction.

Combining scores:
The weighting for the price and the questionnaire should be set before the event. The participants bids are ranked on their score, calculated as:
[Questionnaire weighting] * {Questionnaire score} + [Price weighting] * {Price score} = Overall score.

For example:

You can see in the scenario above that the ‘Percentage leading bid’ method works very well.  Here in round 2, one supplier with better payment terms but a worse price has an equal overall score.  The added value of 90 days payment terms compared to 60 days equates very nicely to 1% of the overall score.

Auction:
During the auction you want to easily see the overall score live and to exactly determine how the weighting is effecting the result.

Here you can see the leading supplier based on the overall score:

And you can also drill into the weighted scoring per bid:

The result is an easy to use weighting mechanism that can be used during a live auction with good visibility of all the related scoring..

Issue with current scoring mechanisms on the market:
Whereas any scoring system will have potential flaws, there are some systems on the market which are more flawed than others and some that are simply incorrect.

The multiplier:

This method uses the questionnaire score to create a price multiplier during the auction.  We find it hard to imagine something so incorrect.  There are many issues with this type of mechanism, which we can only think is a result of a software provider not developing an appropriate mechanism but fudging an exchange rate approach to solve the issue,  such as:

Percentage of leading bid:

For a forward auction, the equation for percentage of leading bid is a simple linear scoring result : ‘Bid divided by Highest bid’.  However, in the public sector we have often seen the inverse of this formula used as the price score calculation in a reverse auction, i.e. Lowest Bid divided by the Bid.  This is mathematically incorrect and actually gives a non linear result, as shown below.  It makes it even harder for the buyer to know how the scoring mechanism would react.

Normalised questionnaire scoring:

We encountered this being used at a council who had applied rigorous OJEU mechanisms to every type of tender, of any value.  In their questionnaire scoring they took the participants’ results and normalised them.  So for example let’s say that the questionnaire is worth 50 points.  Participant A scored 5 out of 50 and Participant B scored 10 out of 50.  After normalisation Participant A gets 25 and Participant B gets 50.  Thus you can see that even if the participant does incredibly badly in the questionnaire, they can still get high points if the other participant has also scored badly.  From the actual points you can imagine how this could drastically change the tender.  In some ways it is saying that there is not room for improvement.  Imagine if the questionnaire was related to quality?  The normalisation could transform a low quality score into a very commendable one, which sadly masks the reality of the situation.

Conclusion:
In summary the mathematics of a weighted tender are actually quite simple and we are proud of the straightforward linear methodology we have developed.  Not only do we believe it is easy to set up but you can also analyse what is going on during an auction.  This is exactly as it should be: fully auditable and compliant.

We are amazed by some of the current mechanisms used in the public sector today. At best they can distort the results in an unpredictable way and at worst are incorrect and non compliant.  Suppliers involved in these tenders might even have the ability to challenge the results which, if you think about it, could create quite a stir.

The biggest disadvantage of these mechanisms is that they are just plain confusing.  Any buyer setting up a tender using these techniques cannot fully grasp how the mechanisms will behave.  And if the buyer will have difficulty, just imagine what the suppliers will think.

Our first upgrade of 2013, with so much more to come!

Over the weekend we completed our first 2013 upgrade, following on from the second upgrade to Samurai back in December of last year.  Such is the elegance of the Software as-a-Service model, each and every one of you can benefit from the upgrade when you next log in.

Quite simply, there are too many improvements to list (70 to be precise), so here are some of the important highlights:

On top of this, we have made a number of upgrades to our infrastructure to keep ahead of the increasing demands of our customers.   We are now sending several thousand e-mails a day, practically becoming a MailChimp in our own right, and so we have upgraded our e-mail account to ensure uninterrupted communications between our Hosts and their respective participants.
We have also upgraded our servers (again) to ensure we maintain our 99.9% uptime record.  Some events have even had over 500 participants involved, and so we’ve learned a great deal about where potential bottlenecks may lie and catered for them accordingly.

What’s next?
We have a number of exciting new features we are working on that will be released imminently.  For example, in collaboration with our customer Arqiva, we are in the final steps of testing and improving our User Hierarchy feature.  This will allow our users to invite friends, colleagues and countrymen to view their sourcing activities, or even to have edit rights should they have a licence.  You can manage all your internal users from a central place, see all the events across your organisation from your own Dashboard, and even invite external users such as consultants to have their input on your activities.

We are also nearing completing of an enhanced white labelling feature with another collaborative partner in the US as we help them with an exciting new venture.  Our users will be able to define their own URL for the sourcing events, such as www.esourcing.marketdojo.com, which will have your logo and menu colours with your own log in, sign up and password reset facilities.  In short, it will be your own portal: even the e-mails will come from you.

Here’s to a fantastic February ahead!

PS:  If there is anything you’d like us to work on, get in touch and we’d be happy to talk.
Market Dojo – e-sourcing made simple

Kontakt

Consultation Call
Vorname*
Nachname*
Firmañía*
Telefonnummer*
Email*
Ihre Wünsche
Lead Gen
Lead Source
Lead Source Detail

Contacto

Consultation Call
Nombre*
Apellido*
Compañía*
Teléfono*
Email*
Me interesa
Lead Gen
Lead Source
Lead Source Detail

Contactez-Nous

Consultation Call
Prénom*
Nom*
Société*
Téléphone*
Email*
Interessé par
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Consultation Call
First Name*
Last Name*
Company*
Phone*
Email*
I'm interested in
Lead Gen
Lead Source
Lead Source Detail

GET IN TOUCH

Test