Isn't using expert consensus opinion just matter of probabilities?

Consider weather forecasts for example. If all models and meteorologist are in close agreement what’s the better bet? That’s they are wrong or that they are right?

Not to say there’s zero chance they all could be wrong but without any other information agreement is still the best bet.

If, for example, local observations are not in agreement that might change the odds but it’s still a matter of probability. Or in the instance of a lone dissenter with a contrary forecast that turns out overturning the science of meteorology, but again, what are the odds?

1 Like

An example, up to the ‘El Faro’ disaster:

All medium term path forecasts of the extra-tropical depression becoming the deadly hurricane Joaquin were false…
…with the exception of the ECMWF forecast, they had it (nearly) right from the beginning.

It was noticeable during the pandemic that weather forecasts here became widely inaccurate. Surrounded by sea our forecasters were deprived of information from aircraft and most FOC shipping is no longer interested in being a VOS.

The ECMWF forecast is a good point.

Experts developing a consensus can be stifling as people are afraid to go against their peers. If encountering a novel situation, such as the increased availability of real time ECMWF data at sea, fear of being an outlier can cause a false consensus not based on the best available data.

I can’t think of any other situations where the expert consensus has turned out to be hilariously wrong, but I’m sure there is one.

1 Like

agreed, its best guess with the available data.
Thats why some of the worlds largest computing installs are crunching the data to work out what might happen next.

In the *The Structure of Scientific Revolutions* Thomas Khun called this “normal science” within which arise “anomalies” which caused old paradigms to be overturned and so forth.

The anomalies were often from new technologies such as the telescope which produced new data that couldn’t be made to fit the old paradigms or models.

I’d call that new realities.

A more precise term is “observable phenomenon”.

Coincidently Dan Little posted about science dealing with anomalies just yesterday.

Scientific rationality and anomaly

“When faced with anomaly, then, the scientist must choose whether to abandon the theory altogether or modify it to make it consistent with the contrary observations. If the theory has a wide range of supporting evidence, there is a strong incentive for salvaging the theory, that is, of supplementing it with some further assumption restricting the application of its laws, or modifying the laws themselves, to reconcile theory with experience”.

.

Looking at spaghetti models of hurricane forecast beyond 72 hrs. or so is a good way to quickly get a sense of the degree of uncertainty in the forecasts in general but it’s not very useful in practice for avoidance planning.

Using a single source the NHC for example in the U.S, and making an assumption about the forecast error (which is what the 1-2-3 rule does) is a better bet than trying to guess which individual forecast is the most accurate. All models will tend to merge as forecast time gets shorter anyway.

Yes indeed, predicted hurricane paths may look largely erratic over days; there are too many unknown variables.

However, in the ‘El Faro’ case, these early mid-term, nearly unanimously false predictions influenced later decisions:

The ‘Bon Voyage’ meteorologists were very hesitant to adapt their short-term forecasts to the real-time positions of Joaquin. It was more a ‘yes, Joaquin is more south than we thought… but it will go north soon’.
Only their last relevant forecast, ready at 23:00 the evening before the end, showed Joaquin slightly south of San Salvador Island (downloaded by El Faro only at 04:45 the final morning…).
Until the tragic end, Joaquin never went northwards.

The captain’s brain was so impregnated with the early mentions of a U-turn to North, that he did not even realize that at every new real-time position, Joaquin was more South than before…

How about how long Ukraine could last against a Russian invasion? All the experts were saying that Russia would defeat Ukraine’s military in less than a month.

He was also so focused on his weather program he didn’t use any other sources, like the most up to date information from NOAA or having the mates plot the storm from the Sat C messages.

Yeah that’s super weird. Guess you can’t trust the experts.

I think you could probably get a better idea of the probabilities if you were able to determine an elasticity of consensus modifier, we’ll call it EOC.

The elasticity of consensus is closely related to the risk of associating oneself with wingnuts, let’s call it AOWW.

Disciplines in which there is a low risk of AOWW receive regular challenges to the consensus, something necessary for any healthy consensus. Questioning short term weather forecast modeling has a low, albeit not zero risk of associating oneself with people who believe the government is controlling the weather with infernal machines. Challenges to the consensus are judged on their merits.

In other disciplines or subgroups of disciplines even slightly deviating from the consensus can quickly associate oneself with people who believe Germ theory is fake and crystals have healing powers. The consensus is brittle because the AOWW risk is high, and therefore weak. The response to novel or developing situations is often untrustworthy.

The highest probability is to follow the guidance of people who don’t acknowledge the AOWW risk and disregard it yourself, because who cares.

This isn’t always easy but it’s a fine battle to fight.

They only started getting aid after they held off Russia’s attack long enough for aid to be delivered. If the experts were right the aid wouldn’t have mattered.

Why are you coming in here shrieking about Ukraine?

If you just want to give ignorant takes about the war start another thread. If you just want to yell at me for wrongthink pm me and I’ll give you my number.

If there’s a lack of consensus the best bet is to withhold judgement. Usually the reason for the lack of consensus can be found with a little digging. For example with hurricane (TC) forecasts in the tropics it’s often weak steering forces.

The uncertainties in TC forecasts are fairly well known. If a non-expert creates a forecast that goes against the consensus but doesn’t use accepted methods, by consulting an oracle for example, the best bet is to disregard it.

You said you couldn’t think of another time the experts had been that wrong and I was giving an example. Why do you have to turn everything into an argument and then complain about arguing?

There are many examples of experts being wrong.

Famously in finance LTCM.

In civil engineering the Tacoma Narrows Bridge. In medicine the practice of “bleeding” patients, the list is very long.

But for someone who has to evaluate an area outside their own experience, for example when sailing I sometimes had to make judgements about marine engineering, it becomes a matter of what is the best bet?

That doesn’t have anything to do with how consensus is developed and maintained. Consensus needs to be regularly challenged to be healthy. This isn’t a lack of consensus but what keeps a consensus valid in the first place.

The various ways in which the strength of steering forces are measured would be a better analogy.