The Corona Virus

[AMX]

But it doesn't really matter, I think, if people do refuse the vaccine. So long as it's available to those who do want it, and particularly to vulnerable people, we can start treating Covid-19 like a normal flu - there'll be no need for all the extraordinary measures, even if it remains endemic.

Please keep in mind that "vulnerable people" includes immune deficient people who can not be vaccinated successfully.

[Peelee]

[Citation Needed]

People have been science denying since the first man made fire. There's no indication that I can tell that we live in a world that is more science-adverse today than 70 years ago. Science literacy by all metrics is up in 2020 world wide compared to 1950 not to mention the 1930's when a vaccine was first made and then dropped.

This is, in fact, so well-understood even among scientists that a Nobel Laureate even formally published it: Planck's Principle, put simply, is that science advances one funeral at a time.

[Tvtyrant]

That was a different world. Most people really believed in science, back then.

But it doesn't really matter, I think, if people do refuse the vaccine. So long as it's available to those who do want it, and particularly to vulnerable people, we can start treating Covid-19 like a normal flu - there'll be no need for all the extraordinary measures, even if it remains endemic.

Anti-vaccine movements have nothing to do with science and everything to do with distrust of authorities and anti-authority sentiments. People don't mistrust science more now than when they were hanging witches, or the 1930s and 1950s. It is when mistrust of authority grows that the movement takes off again; for instance in my home state it was prevalent in the 1970s as well as the 2010s.

[veti]

[Citation Needed]

People have been science denying since the first man made fire. There's no indication that I can tell that we live in a world that is more science-adverse today than 70 years ago. Science literacy by all metrics is up in 2020 world wide compared to 1950 not to mention the 1930's when a vaccine was first made and then dropped.

Science literacy doesn't increase faith in science. Quite the reverse, in fact. In the 1950s, "science" was seen as a slowly but inexorably expanding study, rolling back the frontiers of ignorance. Now it's seen as a tangle of interested parties all pulling in different directions, and all lying about it.

Citation? Page up a bit in this very thread and you'll see people, including me, stating their hostility to the vaccines available now because they (we) don't trust the science behind them. It's not "our" science. But when a vaccine is available that I would be willing to trust, others will continue to refuse it on exactly the same grounds, purely because they rely on different news sources.

And there's no longer a single news source, in most countries, that's trusted by an overwhelming majority of the populace. So no way to "authoritavely" say "yes, this is our vaccine".

[BeerMug Paladin]

I think it should be fairly non-controversial to say that a medical professional working in a related field is better to recommend a vaccine than a... Anyone else. Whether we learn about vaccines from the former instead of the latter is likely to have a big influence on the general level of trust among the public. I haven't paid much attention to the news of vaccine development, but I'm not terribly confident that those anyone else's won't be left out of the decision making process of what gets called a vaccine. At least in my region of the world.

Also, I feel it should be said that no matter the field of science, you can always find some kook specialized in that field who is willing to promote ideas in total opposition to the generalized scientific consensus within that field. It seems crazy, but those folks are out there if you're motivated enough to find them. But the solution to that is simple, wait to see what the broad scientific community says, not just one individual. (Which again, means that in order to trust a vaccine, a lot of people who place a high value in science are going to wait until they see that.)

[Razade]

Science literacy doesn't increase faith in science. Quite the reverse, in fact. In the 1950s, "science" was seen as a slowly but inexorably expanding study, rolling back the frontiers of ignorance. Now it's seen as a tangle of interested parties all pulling in different directions, and all lying about it.

I reject this concept. Understanding science increases, so does the faith in it. It's demonstrable. People that are modern conspiracy theorists are no different (and a few of them far older than most realize) than the same people who thought they could stop masturbation by banning spicy foods and making people eat graham crackers. These people don't have a lack of faith (I think we really should be using confidence instead of faith) in science. They have a lack of confidence in reality because it doesn't gel with how they want it. A really really good discussion on this that I can't get into because of board rules was covered by Folding Ideas "In Search of the Flat Earth". I recommended everyone watch it.

I also recommend looking into Abbie Richards and her work on the Conspiracy Chart. Both are invaluable and demonstrate exactly the contrary to what you're espousing veti. It's not science denialism that's the issue here. It's something far different that I really sadly cannot get into on these boards.

Citation? Page up a bit in this very thread and you'll see people, including me, stating their hostility to the vaccines available now because they (we) don't trust the science behind them. It's not "our" science. But when a vaccine is available that I would be willing to trust, others will continue to refuse it on exactly the same grounds, purely because they rely on different news sources.

A few forum goers on an obscure website isn't a citation. It's anecdote.

[catagent101]

I reject this concept. Understanding science increases, so does the faith in it.

Science, unfortunately, says otherwise. And I mean it makes a certain sort of sense. Antivaxxers do not live in caves and hunt mammoths in the snowy wastes. They had to learn their subatomic particles, their soh and coh and toa, and probably have heard the story of the guy who made the first vaccine by infecting people with harmless cowpox to protect against smallpox. And probably forgot a good bit of it since it wasn't relevant to their job, but they are perfectly aware that evidence against their beliefs exists.

I think you kinda pointed out the flaw in your argument yourself. For some people, some things trump facts. Sometimes it is as simple as wanting to avoid being ostracized, sometimes it is more complex as the video talked about (I recommend it too btw).

What I personally suspect is that once the vaccine(s) is a real physical thing and the inevitable campaign to vaccinate everyone starts, the vaccination rates will be significantly higher than the current trust statistics predict. Particularly if people see it as the ticket back to normalcy, which currently seems to be the case. It seems like a pretty easy sell and I don't think the antivax movement has anywhere near enough clout to stop it. Of course, if politics enters the ring, that may not work out as well.

[Forum Explorer]

I'm curious if we'll see businesses make getting the vaccine mandatory before their employees are allowed back to work. Which I'm pretty sure they are allowed to do, after all, there's a lot of animal jobs out there that require you to have both rabies and tetanus shots up to date before you can work.

[Razade]

Science, unfortunately, says otherwise. And I mean it makes a certain sort of sense. Antivaxxers do not live in caves and hunt mammoths in the snowy wastes. They had to learn their subatomic particles, their soh and coh and toa, and probably have heard the story of the guy who made the first vaccine by infecting people with harmless cowpox to protect against smallpox. And probably forgot a good bit of it since it wasn't relevant to their job, but they are perfectly aware that evidence against their beliefs exists.

The first study cited only found a small negative based on...not a massive sample size picked not by science literacy but by "world view" and not by their level of education or any actual relevant metric. It checked for political affiliation (which we can't discuss more here) and other cultural aspects. Even then, the second study with more refined parameters removed that small negative effect and concluded something different. Taken from the second survey

"Doing so reveals that there is in fact little disagreement among culturally diverse citizens on what science knows about climate change. The source of the climate-change controversy and like disputes is the contamination of education and politics with forms of cultural status competition that make it impossible for diverse citizens to express their reason as both collective-knowledge acquirers and cultural-identity protectors at the same time. "

Which backs up the rest of why I just outright deny veti's assertion. It isn't about denying the science, it's not about a lack of faith in science. It's about conforming to a conspiracy because if Climate Change is real, that means the rest of the cultural mores of that conspiracy theory cannot be correct and that doesn't gel with the world view. They have to, despite the wider world rejecting their conspiracy, deny the science and engage in massive cognitive dissonance.

This is further backed up by Pew Research Group (a far better source than Slate) that only confirms this further. This was published this year as well so the information is not only from a wider sample pool than the 1,500ish people the Slate studies linked cover by more than half. Pew's was almost 4,000 people with surveys stretching over a far longer period of time than the ones cited by Slate. Over a three to four year period. Their method also brings into consideration education level. It's a newer, better operated, more broadly operated and longer operated study.

The confidence that Americans (because that's who they survey) have in science is up. So is science literacy. You can look for yourself. . I'll reiterate this was a broader, longer, and more recent study. Confidence in science and scientific literacy are both on are rise. Not one falling while the other rises. They're both rising together. Now, that doesn't mean there is a correlation but what it absolutely does blow out of the water is that scientific literacy has a negative effect on the over all population's confidence in science. It seems it's the inverse.

The conclusion of the Pew Research survey drives this home. More people say they have more confidence in research and science the more the data is shared and freely discussed. Improving access and understanding of data and research increases, outright, the general trust in science. By a wide margin.

You're also...way over-estimating most anti-vaxxer's level of science education. No one's claiming they're cave people but most of them aren't doctorate level in the sciences either.

[I think you kinda pointed out the flaw in your argument yourself. For some people, some things trump facts. Sometimes it is as simple as wanting to avoid being ostracized, sometimes it is more complex as the video talked about (I recommend it too btw).

I don't particularly find being honest that there are outlier cases and exemptions to the overall rule to be a flaw in an argument. It's pointing out that there are outliers and you'd expect that. Firstly, it's good manners to offer up a falsifier for your own argument because an argument that can't be falsified into an argument worth engaging with and secondly it's just a fact. There are people who are alive today that don't trust science. But it's not a flaw in my argument to point out that there are people who deny science. They're a minority. They're fringe groups that are outside the norm. I never said that everyone's confidence in science increases with their science literacy, just that it's not true that an increase in science literacy does not increase confidence in science.

Because it does over all. Yes. Again. There are outliers. That's not indicative to the larger trend that your second, better, study concludes and mine just outright demonstrates. Their science denialism is not because of scientific literacy. It's because they hold fringe views that require them to deny science to prop up other ideas which is not a point in veti or your argument's favor.

[Grey_Wolf_c]

...

Well, this is a confluence of my job and the pandemic I was not expecting to see: reportedly, the UK "missed" 16000 covid patients because they were using one patient per column in an excel sheet, and they hit the maximum number of columns. (My guess is they get the list of names in a comma-separated list, and import it directly to excel)

Did they switch to a better data storage, such as, say, a database? No

Did they pivot the input so it's at least one row per patient and have room for a million names instead of 16k? No.

OK, so did they use a second row in the same sheet? No.

What did they do, then? Split the data across multiple excels.

Words fail me.

Grey Wolf

[Keltest]

...

Well, this is a confluence of my job and the pandemic I was not expecting to see: reportedly, the UK "missed" 16000 covid patients because they were using one patient per column in an excel sheet, and they hit the maximum number of columns. (My guess is they get the list of names in a comma-separated list, and import it directly to excel)

Did they switch to a better data storage, such as, say, a database? No

Did they pivot the input so it's at least one row per patient and have room for a million names instead of 16k? No.

OK, so did they use a second row in the same sheet? No.

What did they do, then? Split the data across multiple excels.

Words fail me.

Grey Wolf

Im sorry, i seem to be afflicted with early morning groggyness. I could have sworn you just posted that a massive government entity was running their data collection efforts on a life threatening disease like a high schooler who is still figuring out how to store lots of numbers in a way thats legible without spending money on something they dont care about that much. But that cant be right.

[Grey_Wolf_c]

Im sorry, i seem to be afflicted with early morning groggyness. I could have sworn you just posted that a massive government entity was running their data collection efforts on a life threatening disease like a high schooler who is still figuring out how to store lots of numbers in a way thats legible without spending money on something they dont care about that much. But that cant be right.

It is... unclear if this excel is the ultimate data collection system, or it is some form of step to feed it. Professionally speaking, I'm not sure which of those options would be worse. There is a comment about "dividing the data into batches" (the source for my "multiple excels" line) that suggests this is a data-consuming step. But that just raises so many more questions, like "why not skip excel and feed the csv directly to the end system, which if it can read excel, can read csv" and "how much are these people paying to do this, because they can't be paid for this, right?" and "can we dust off the death penalty laws for extreme incompetence?"

Grey Wolf

[Keltest]

It is... unclear if this excel is the ultimate data collection system, or it is some form of step to feed it. Professionally speaking, I'm not sure which of those options would be worse. There is a comment about "dividing the data into batches" (the source for my "multiple excels" line) that suggests this is a data-consuming step. But that just raises so many more questions, like "why not skip excel and feed the csv directly to the end system, which if it can read excel, can read csv" and "how much are these people paying to do this, because they can't be paid for this, right?" and "can we dust of the death penalty laws for extreme incompetence?"

Grey Wolf

If its the latter, i could maybe see to letting them off on the understanding that it was probably done by a couple of poor guys up at 2 AM desperately trying to make a system that functions (i refuse to say works) when they arent getting paid nearly as much as they should be, with the knowledge that lives may literally be hanging on their ability to get this thing out as quickly as possible. By the time you get to 2 AM development, nothing makes sense anymore anyway. But if its the former, i think the only professional response would be to strap their computer to their back and send them into exile. Nobody who isnt powered by an unhealthy amount of coffee should look at this and go "yes, this is what we want."

[snowblizz]

Im sorry, i seem to be afflicted with early morning groggyness. I could have sworn you just posted that a massive government entity was running their data collection efforts on a life threatening disease like a high schooler who is still figuring out how to store lots of numbers in a way thats legible without spending money on something they dont care about that much. But that cant be right.

This is exactly how it works in the real world. Databases are hard (expensive), Excel is easy (cheap). Hilarity ensues.

The NHS some years ago was still stuck on Windows XP computers and large part of the system got locked down by a virus. Because it costs money to upgrade and hire nonlazy techstaff.

It's quite common. Your primary system wasn't built to handle some special data form an exceptional circumstance. So you muddle about in Excel because it lets you do anytihng you want, though not necessarily very well.

[Grey_Wolf_c]

This is exactly how it works in the real world. Databases are hard (expensive), Excel is easy (cheap). Hilarity ensues.

Not really? Leaving aside that MySQL is literally free, the cost of an excel licenses versus a SQL server license for a government is not even a rounding error on their overall costs. What is expensive is the people that you need to run & code them, whether it be the excel or the DB.

But in this case, what is almost certainly the most expensive bit is the actual manager of those poor bastards, who insisted on an excel step (based on past experience, "because we've always done it that way, and I'm comfortable with it") and refused to listen to warning such as "maybe we shouldn't" and "what about when there are more than a handful of names in the list" and "how about we put one name per row instead, it'll only take a minute".

Grey Wolf

[Tyndmyr]

Im sorry, i seem to be afflicted with early morning groggyness. I could have sworn you just posted that a massive government entity was running their data collection efforts on a life threatening disease like a high schooler who is still figuring out how to store lots of numbers in a way thats legible without spending money on something they dont care about that much. But that cant be right.

I have some really bad news for you about what happens to software development/data science projects once you get enough fingers into the pot.

Often the people executing it are fully aware that it's a terrible idea on any technical basis, but the decision was made at a level where they weren't involved in it.

[Manga Shoggoth]

Not really? Leaving aside that MySQL is literally free, the cost of an excel licenses versus a SQL server license for a government is not even a rounding error on their overall costs. What is expensive is the people that you need to run & code them, whether it be the excel or the DB.

But in this case, what is almost certainly the most expensive bit is the actual manager of those poor bastards, who insisted on an excel step (based on past experience, "because we've always done it that way, and I'm comfortable with it") and refused to listen to warning such as "maybe we shouldn't" and "what about when there are more than a handful of names in the list" and "how about we put one name per row instead, it'll only take a minute".

Grey Wolf

EDIT: Just re-read the starting comment about Maximum Number of Lines in Excel. Yup - that is stupid. The last time I saw that was in in the late '80s, and they at least had the sense to switch to a database then.

However...

If the office is using MS Office or its variants, they already have Excel licenced, and probably already have staff with some idea of how to use it.

In that case they don't need to spend money on someone who can use brand X database technology (or training up someone for it), let alone people with enough knowledge to secure it properly properly (although many managers won't think of that either).

Databases are not magic. Getting a database - and an appropriate front end - set up properly is hugely expensive, and that's before you involve the political effort getting technical people from outside the department.

These things may look like a rounding error from your point of view, but it almost certianly isn't from thiers. And even if it isn't much they don't want the extra expense on their cost code.

Used properly Excel is an excelent tool. I've been using it since Version 5. Used badly, it's a mess (and believe me, I have seen some amazing messes), but the same can be said for databases as well.

Side note: When we were working on Y2K preperation in the early to mid '90s we were swapping spreadsheets around because Excel was the only thing we could guarantee that everyone in the Company would have access to. I did have access to both Sybase and Access at the time so I could quite easily extract the data from spreadsheets and collate the data into a database, but extracting and redistributing responses became a lot more awkward.

EDIT the Second: Just seen the write-up on The Register (A UK Technical News website that isn't always overly serious). It's grimly hilarious - I'd link it here, but it may be a tad political.

However, I'd be very wary of taking technical news from The Daily Mail - it is a tabloid, and tends to put sensation over accuracy in its writing.

[Grey_Wolf_c]

If the office is using MS Office or its variants, they already have Excel licenced, and probably already have staff with some idea of how to use it.

In which case they also likely have an Access license, if not a full-blown MS suite license with access to all of those plus SQL Server. In fact, I'd be astounded to hear any organisation the size of the UK government doesn't have SQL Server or some other DB in their data departments. This idea that the UK only has licenses for excel is too ludicrous to contemplate.

More importantly, from what details they've revealed, the issue is not that they don't have a DB, but that they forced the data loading to go through excel, and executed the process so poorly and with such little care that they 1) didn't consider such basic steps as "replace the commas with carriage returns so that it's one name per row" or equivalent pivot and b) they didn't even set up such basic error-control as to "count how many names/commas go in, count how many go out", which is what pushes this debacle from "an uups" to "fire-worthy incompetence", as far as I am concerned.

In that case they don't need to spend money on someone who can use brand X database technology (or training up someone for it), let alone people with enough knowledge to secure it properly properly (although many managers won't think of that either).

Those are one and the same. These days Excel is so connected to SQL Server and Access that anyone hired "to do excel data loading" is going to also know SQL. Heck, 99% of the time I encounter excel it is as front end or data loading for SQL Server.

Databases are not magic. Getting a database - and an appropriate front end - set up properly is hugely expensive, and that's before you involve the political effort getting technical people from outside the department.

No, they are not hugely expensive. The people with the ability to run them are expensive, but using excel as a DB is equally expensive as using MySQL or Oracle or SQL Server, because it takes the same exact people to do any of them. And if you go without, and are as big as a government, it is even more expensive in terms of hours lost as people try to make sense of data in a gazillion excels across the company. Not, again, that I believe for a second that the UK doesn't have actual DBs.

Side note: When we were working on Y2K preperation in the early to mid '90s we were swapping spreadsheets around because Excel was the only thing we could guarantee that everyone in the Company would have access to. I did have access to both Sybase and Access at the time so I could quite easily extract the data from spreadsheets and collate the data into a database, but extracting and redistributing responses became a lot more awkward.

Database reporting tools have improved significantly over the last 25 years, and a large number of free fairly decent ones exist and are widely used. Excel is still used, but there is no need to extract & distributing them - everyone can just connect to the relevant DBs with pre-canned SQL statements and just refresh the data as needed.

Grey Wolf

[snowblizz]

In which case they also likely have an Access license, if not a full-blown MS suite license with access to all of those plus SQL Server. In fact, I'd be astounded to hear any organisation the size of the UK government doesn't have SQL Server or some other DB in their data departments. This idea that the UK only has licenses for excel is too ludicrous to contemplate.

More importantly, from what details they've revealed, the issue is not that they don't have a DB, but that they forced the data loading to go through excel, and executed the process so poorly and with such little care that they 1) didn't consider such basic steps as "replace the commas with carriage returns so that it's one name per row" or equivalent pivot and b) they didn't even set up such basic error-control as to "count how many names/commas go in, count how many go out", which is what pushes this debacle from "an uups" to "fire-worthy incompetence", as far as I am concerned.

Database reporting tools have improved significantly over the last 25 years, and a large number of free fairly decent ones exist and are widely used. Excel is still used, but there is no need to extract & distributing them - everyone can just connect to the relevant DBs with pre-canned SQL statements and just refresh the data as needed.

Again you fail to comprehend the reality of how things work. You expect everyone to act like they are IT technicians with loads of time to plan a project. In a normal time this would have been a multi-year project involving all kinds of experts. None this was what was available in a chaotic pandemic situation. I am willing to be that the people at the bleeding edge trying to register information were nurses and healthcare workers whose jobs normally does not consist of shuffling data around.

Basically, this has all the hallmarks of Corona rushjob. Systems that were never intended to cope or deal with this have been pressed into service to be used by people who aren't really the ideal ones to be using them in the first place. Oh and their main function is being overloaded too, leaving absolutely no time to do something reasonable about it all.


In fact, I'll add that I would not be at all surprised if somewhere there is a central reporting management system being built that would have alleviated all these issues but it is underfunded, over time, being tweaked because it can not have any security breaches with health data (and was thus delayed) and politicians are endlessly debating whether to axe it or finish it because it is clearly of no use what so ever.

An organisation like the NHS is not like Google. They don't have the money or people to implement whatever quickly when needed. Papering over cracks with Excel sheet is exactly how issues tend to be handled in tech-poor organisations. Even when better solutions may even be available.

[Grey_Wolf_c]

Again you fail to comprehend the reality of how things work.

No, I think it is quite the other way around. But I'm not going to have this discussion with you.

GW

[Manga Shoggoth]

In which case they also likely have an Access license, if not a full-blown MS suite license with access to all of those plus SQL Server. In fact, I'd be astounded to hear any organisation the size of the UK government doesn't have SQL Server or some other DB in their data departments. This idea that the UK only has licenses for excel is too ludicrous to contemplate.

The licencing for Access fluctuates wildly - it even used to be in a seperate product (Office Professional rather than Office Standard, which was just Word, Excel and Powerpoint).

More importantly, from what details they've revealed, the issue is not that they don't have a DB, but that they forced the data loading to go through excel, and executed the process so poorly and with such little care that they 1) didn't consider such basic steps as "replace the commas with carriage returns so that it's one name per row" or equivalent pivot and b) they didn't even set up such basic error-control as to "count how many names/commas go in, count how many go out", which is what pushes this debacle from "an uups" to "fire-worthy incompetence", as far as I am concerned.

Oh, absolutely - add to this using an older version of Excel (I recommend looking up the Register article - I suspect you might enjoy it).

Those are one and the same. These days Excel is so connected to SQL Server and Access that anyone hired "to do excel data loading" is going to also know SQL. Heck, 99% of the time I encounter excel it is as front end or data loading for SQL Server.

No, they are not hugely expensive. The people with the ability to run them are expensive, but using excel as a DB is equally expensive as using MySQL or Oracle or SQL Server, because it takes the same exact people to do any of them. And if you go without, and are as big as a government, it is even more expensive in terms of hours lost as people try to make sense of data in a gazillion excels across the company. Not, again, that I believe for a second that the UK doesn't have actual DBs.

But this really isn't true. It's (sometimes passable) Excel skills and bog all else, and frequently a high-level imperetive that no local databases are set up (due to privacy laws, security concerns and a lot of other things).

The DBA experience is usually with the external contracting companies because the whole point of outsourcing is to get rid of those nasty expensive technical people. Very, very few of the actual staff will have access to this kind of thing.

[Ashtagon]

I have worked in UK government offices, so I have some idea of what is technically available.

Excel and Access are everywhere (as well as Word and Powerpoint). Pretty much everyone will know Word. If you do presentations, you'll probably know powerpoint, although its mostly seen as a manager's toy. Pretty much everyone will know Excel well enough to do data entry and navigate around the user interface, and most people will be able to call on an expert who knows Excel a lot better than that. Access is generally about as well understood as the composition of Neptune's core. Realistically, if an actual database is going to be used (whether Access or a "proper" db), it will require external expertise.

From what I've seen of actual commissioning of databases by external contractors (not government work this time; one was a customer db for a cruise company; one was a room booking diary), the lead time to decide on the technical specifications of what it needs to do once the contract has been granted is about six months, then a further six months for the first working prototype. A final live working version is normally 2-3 years down the line. In places where I have developed databases for employers (using Access), the lead time from initial brief to documented working tool is normally 2-4 months. However, that has obviously been for a far smaller task, and I didn't have to consult with multiple offices across multiple organisations nationwide. It entailed multiple meetings with the end users to discuss their needs, plus a few training sessions once it was ready to go live.

The UK is barely six months into its overall pandemic response. The idea that a database will be been developed in that time is, frankly, crazy talk. While ideally they will be working to build a database (and it is possible that a pre-existing pandemic planning db had been set up back when the NHS had responsibility), I'd be astonished if the private companies who were contracted had one ready to go, or even in development, on day one.

It's probably running on spreadsheets all the way through right now.

[Keltest]

I feel like something needs to be clarified here. The problem isnt that its happening for no reason. Almost nothing happens for no reason at all, people are rarely that arbitrary. The problem is that the only potential reasons for it happening that way are bad, dumb reasons where one or more persons actively ignored common sense and/or expert opinion to do this in probably the least effective way possible, BECAUSE of those bad dumb reasons. Even if Excel was the only tool available, even if there were little virus golems holding guns to peoples heads forcing them to use Excel for this task, they still could have implemented it better. I mean seriously, losing data because you formatted your spreadsheet poorly? This is incompetence, plain and simple.

[Manga Shoggoth]

I have worked in UK government offices, so I have some idea of what is technically available.

Excel and Access are everywhere (as well as Word and Powerpoint). Pretty much everyone will know Word. If you do presentations, you'll probably know powerpoint, although its mostly seen as a manager's toy. Pretty much everyone will know Excel well enough to do data entry and navigate around the user interface, and most people will be able to call on an expert who knows Excel a lot better than that. Access is generally about as well understood as the composition of Neptune's core. Realistically, if an actual database is going to be used (whether Access or a "proper" db), it will require external expertise.

Thank you for the internal view - I'm looking from this from the position of one of the outside contractors. About half my life has been working on Government projects. Not, thankfully, the NHS - that is (for a variety of reasons) a whole different ball of complex.

Oh, and I like "Access is generally about as well understood as the composition of Neptune's core"...

I feel like something needs to be clarified here. The problem isnt that its happening for no reason. Almost nothing happens for no reason at all, people are rarely that arbitrary. The problem is that the only potential reasons for it happening that way are bad, dumb reasons where one or more persons actively ignored common sense and/or expert opinion to do this in probably the least effective way possible, BECAUSE of those bad dumb reasons. Even if Excel was the only tool available, even if there were little virus golems holding guns to peoples heads forcing them to use Excel for this task, they still could have implemented it better. I mean seriously, losing data because you formatted your spreadsheet poorly? This is incompetence, plain and simple.

Absolutely. My issue here is that Excel, used properly, is a very useful tool. But it has its limits, and like every tool it can be misused, abused and otherwise snarl things up. After all, a fool with a tool is still a fool.

Simply repeating "should have used a database", which is the default reaction, is not really all that helpful.

Incidentally, for those who would like to read a good rant, El Reg has just published this one, which I think is sufficiently unpolitical for me to link here: Excel Hell: It's not just blame for pandemic pandemonium being spread between the sheets.

[Rogar Demonblud]

Anyway, back to the old topic.

The WHO announced earlier today that the confirmed case count globally has passed 10% of the population. And we are almost certainly undercounting because asymptomatic people aren't getting tested.

[warty goblin]

Honestly the thing that baffles me most about the Excel thing isn't that they used it. Its that apparently they made a unique column for each case. Which isnt wrong per say, its just deeply, deeply weird since the longstanding convention for flat data files is variables on columns, cases on rows. It's like if someone decided their specific town was going to have green traffic lights mean stop, and red mean go.

[Grey_Wolf_c]

Honestly the thing that baffles me most about the Excel thing isn't that they used it. Its that apparently they made a unique column for each case. Which isnt wrong per say, its just deeply, deeply weird since the longstanding convention for flat data files is variables on columns, cases on rows. It's like if someone decided their specific town was going to have green traffic lights mean stop, and red mean go.

Based on the sparse descriptions of the issue, I'm almost certain that this is a botched csv load. Someone somewhere generates a comma-separated list of names, sends them to someone else who load them into excel and then pushes them somewhere else (based on the word "batch" in the article). When they did it the first few times, with a few dozen or hundred names, it worked. As the numbers increased to beyond 16k, names started to be dropped unnoticed. I also strongly suspect that these lists of names are coming from multiple places, and someone was just copy-pasting them into a single unified list, since their "solution" is to load them separately.

GW

[Ibrinar]

I had a look at excess mortality stats, in the countries I normally check it isn't too extreme. But Peru and mexico for instance get kinda crazy . https://www.ft.com/content/a2901ce8-...c-cbdf5b386938 says Peru is at 2500 per million excess deaths (they are nearing 1000 per million in the covid statistics). I guess we really will only get decent numbers for the world wide effect in retrospect.

Anyway, back to the old topic.

The WHO announced earlier today that the confirmed case count globally has passed 10% of the population. And we are almost certainly undercounting because asymptomatic people aren't getting tested.

I would assume when they are making an estimate they would include asymptomatic in it? Depends on how they made the estimate I guess.

[Tyndmyr]

Again you fail to comprehend the reality of how things work. You expect everyone to act like they are IT technicians with loads of time to plan a project. In a normal time this would have been a multi-year project involving all kinds of experts. None this was what was available in a chaotic pandemic situation. I am willing to be that the people at the bleeding edge trying to register information were nurses and healthcare workers whose jobs normally does not consist of shuffling data around.

I mean, I've built a large scale data ingest into database system in...three days, working entirely solo. This isn't a technically strange thing, or something with undue requirements. Things like checking to ensure that the number of records are the same on both sides is...really basic.

It isn't that there isn't enough time. It's that decisions were made poorly. This happens an awful lot, sure, but it doesn't really excuse it. The delays are not caused by technical issues, but by beauracracy. Emergencies such as a pandemic should be a time to relax such rules, not to insist on train wrecking the data out of blind adherence to them.

Based on the sparse descriptions of the issue, I'm almost certain that this is a botched csv load. Someone somewhere generates a comma-separated list of names, sends them to someone else who load them into excel and then pushes them somewhere else (based on the word "batch" in the article). When they did it the first few times, with a few dozen or hundred names, it worked. As the numbers increased to beyond 16k, names started to be dropped unnoticed. I also strongly suspect that these lists of names are coming from multiple places, and someone was just copy-pasting them into a single unified list, since their "solution" is to load them separately.

GW

Almost certainly. This has the hallmarks of something dumped on an intern.

[Rogar Demonblud]

I would assume when they are making an estimate they would include asymptomatic in it? Depends on how they made the estimate I guess.

It's not an estimate. They added up all the reported cases.