|
Currently, I'm working on improving my lecture notes for an introductory OR/MS class targeted toward 1st year MSc students in engineering management with no background in OR. Before getting to the technical parts such as modeling and solution techniques, I usually spend about one up to two weeks on the fundamental concepts and topics such as OR origin and history, importance, application areas, and of course relevant success stories. Right now, I'm interested in introducing students to some of the pitfalls and risks associated with OR practice in real world in order to warn them about what could possibly go wrong during various problem solving stages such as problem definition, data gathering, problem abstraction, model building, model verification and validation, solution development, and finally and more importantly solution implementation. However, my initial search results are not very promising. The closest material I've seen on the web about this issue is an article in Wikipedia about model risk. Your personal thoughts on this matter or any relevant content on the web such as papers, books, or case studies of failure (not success) stories is highly appreciated. |
|
There are many pitfalls in the whole process, but to my experience one of the biggest is data. Do we have data, electronically available, correct and consistent? The picture below is not a joke. Our partner said, "we will send you the data", well: they did!!
Other pitfalls: 1. You talk to the wrong people. Those who are happy about what you do (and who talk to you) are not the same who decide about the money. 2. Wrong expectations. Some think that maths and a computer can solve everything, often not true. 3. No or too many goals. Sometimes there are so many shareholders and so many objective functions that finding a "good" solution is not at all well defined. Thanks, Marco.
(Feb 16 at 10:09)
Ehsan ♦
@Marco, still scanning with your phone ?
(Feb 16 at 10:19)
Bo Jensen ♦
4
@Bo, you wouldn't believe (or maybe you would). These are screen shots from HTML data, they printed it, they stapled it. This is two days work! And then we went to their IT guys and asked about their data base and they said, one cannot extract that data from it. Sometimes, some SQL statements come in handy ;-)
(Feb 16 at 11:09)
Marco Luebbecke
@Marco, having seen the above picture, I believe anything can be expected, that's definitely an insane show-stopper :-)
(Feb 16 at 14:01)
Bo Jensen ♦
1
The main problem about retrieving data in many organizations is that people responsible for maintaining databases are either not the people who have designed them or not familiar with technical requirements (a third choice would be just laziness). I was involved in a consulting project in which the client had all of their processes and information stored in a PowerDesigner database. We asked them to give us the necessary information using a simple report procedure. They gave us something that it made us to enter all the necessary information manually based on the hard copies of their reports.
(Feb 16 at 14:15)
Ehsan ♦
yes. but I must say that this was the beginning of a truely wonderful project!
(Feb 16 at 14:34)
Marco Luebbecke
Agree that time to deal with data is usually the biggest unknown at the start! I hate it when the business people I talk to say that their data is good quality, so I give an estimate based on that, and then when the data comes through it has all sorts of subtle issues that need detection and repair.
(Feb 16 at 16:12)
DC Woods ♦
showing 5 of 7
show all
|
|
One of the biggest risks in practice is that the problem the client asks you to solve isn't actually what they need solved. Sometimes don't really know what they need, but they have an idea - perhaps something they've read about or seen done somewhere else, or just plain assumption. It is important to elicit from the client what they are actually trying to achieve with the project, and not just perform a checklist of tasks they give you. There is the old story of the consultant who was called in to optimize the elevator logic in a tall building. People had been complaining that the wait times were too long. After analysing the arrivals and destination patterns and performing various simulations the consultant concluded that he couldn't actually improve the wait times much. Instead he recommended putting up mirrors in the waiting areas. People were then too busy surreptitiously eyeing each other up to notice the wait and the complaints stopped. Thanks David. Actually, your post and its example serve two important issues. First one is the importance of correct problem definition, which is the problem is not always what the client think it is. Second is the necessity of having a multi-disciplinary approach to OR problems (in the example, psychology is saving the consultant).
(Feb 16 at 08:34)
Ehsan ♦
3
Another reality is that the notion of 'the problem' may be a fallacious trap. Behind every OR initiative is usually an urge to improve existing conditions. But there is rarely unanimity about what's wrong with existing conditions, or indeed, what those 'existing conditions' are. In the elevator example, the 'existing condition' is the too-high number of complaints. Business owners care about that. It isn't sub-optimal elevator logic, the OR expert's natural frame. Add a layer of unfamiliarity due to the expert being an outsider, and we're primed for miscommunication and failure.
(Feb 24 at 14:38)
SanjaySaigal
|
|
Good suggestions by others, I will just add one thing. Often the the simplicity is overlooked, I mean we study for many years in advanced methods and really want to show our worth and apply these methods IRL. A number of practical problems may not be so difficult and making a overly complicated set up can spook non OR people (decision makers), start simple and then build it up later i.e KISS. |
|
There is a wonderful article by Jonathan Flowers entitled Wearing your Client's Shoes, published in OR Insight 13, 15–21 (1 July 2000) | doi:10.1057/ori.2000.13 http://www.palgrave-journals.com/ori/journal/v13/n3/pdf/ori200013a.pdf It isn't totally what you are talking about, but great to help students get a bit of managerial context. I'll contact Jonathan and see if he will let me provide a pdf on my site: www.learnandteachstatistics.wordpress.com |
|
A great source of stories of both successes and failures (and failures turned into successes) in the practice of OR is the old Fifth Column feature from Interfaces from back in the 1970a up until a few years ago (is he still at it?). Say what you will about Gene Woolsey, he's a great storyteller. Thanks Matthew. This is a really good source. A quick search on Interfaces website shows that Gene is still active (73 results with two new ones in 2010 and 2011).
(Feb 16 at 13:55)
Ehsan ♦
|
|
It's easy to accept constraints at face value. If you ask the reason for each constraint, you'll sometimes find out it's because "that's how we've always done it" or because someone thought it should be that way intuitively. Thanks Paul. I remember one of my professors telling us in a systems theory course that many of the barriers (similar to constraints in OR) in a system are in fact mental barriers, not actual ones. This advice surely applies to OR practice.
(Feb 17 at 01:38)
Ehsan ♦
|
|
What a great topic! A few OR pitfalls: In general we fall in love with our methods or specialties. That's not always a good thing. It's not necessary to do a thing just because we can. I have experience as an optimization guy and as a statistics guy. Neither of these skill sets may be the right one for a given problem. Sure, I can do some pretty cool scheduling. Maybe the actual problem calls for something much simpler and more robust. Data is not trustworthy. Even if we have accurate data, we don't necessarily have relevant data. Or the data we have may not have anything to do with the problem we're solving. I have seen instances in which the analyst says he needs certain data to solve a problem. "We don't have that data exactly," says the customer. "Use this instead." If an analyst goes along with this approach, he just committed OR malpractice. If you want more cautionary tales I suggest you talk to practitioners rather than academicians. Good luck. |
