Stack Overflow made the BBC news - Copycat coders create 'vulnerable' apps

Question

• Is this an attack on Stack Overflow?

• Q&A sites?

• GitHub?

• The Internet?

What are we doing here?

Does it benefit anyone?

Do we have a moral conscience/responsibility to think and act about these things (or do the best we can with the time we allocate to helping programmers in distress)?

Before we start I have a 7,500+ rep on Stack Overflow, jon-goodwin. I expect you to express your opinions and views/values since the tags are discussion & news! I must admit the change in mindset required from Stack Overflow main site to the meta.stackexchange.com was a strain on my brain and nerves but I think I'm getting the hang of it.

Stack Overflow made the BBC news:

Lazy developers who copy solutions to tricky programming problems are creating apps that are vulnerable to attack, research suggests.

A team of computer scientists looked at more than 72,000 chunks of code found on the Stack Overflow website.

[…]

But researchers found many of the most copied snippets lacked basic checks that would stop common attacks.

The dangerous code chunks often used obsolete functions, did little to check user responses and did not look for attempts to break the application, said the study.

[…]

The paper

An Empirical Study of C++ Vulnerabilities in Crowd-Sourced Code Examples

Morteza Verdi, Ashkan Sami, Jafar Akhondali, Foutse Khomh, Gias Uddin, Alireza Karami Motlagh (Submitted on 3 Oct 2019)

Software developers share programming solutions in Q&A sites like Stack Overflow.

The reuse of crowd-sourced code snippets can facilitate rapid prototyping. However, recent research shows that the shared code snippets may be of low quality and can even contain vulnerabilities.

This paper aims to understand the nature and the prevalence of security vulnerabilities in crowd-sourced code examples.

To achieve this goal, we investigate security vulnerabilities in the C++ code snippets shared on Stack Overflow over a period of 10 years.

In collaborative sessions involving multiple human coders, we manually assessed each code snippet for security vulnerabilities following CWE (Common Weakness Enumeration) guidelines.

From the 72,483 reviewed code snippets used in at least one project hosted on GitHub, we found a total of 69 vulnerable code snippets categorized into 29 types. Many of the investigated code snippets are still not corrected on Stack Overflow. The 69 vulnerable code snippets found in Stack Overflow were reused in a total of 2859 GitHub projects. To help improve the quality of code snippets shared on Stack Overflow, we developed a browser extension that allow Stack Overflow users to check for vulnerabilities in code snippets when they upload them on the platform.

I find interesting:

Prof Sami said the team had developed an extension for the Chrome browser that checks when code is copied from Stack Overflow and lets coders know if it is poorly written or insecure.

The most widely used insecure code blocks turned up in more than 2,800 separate projects on the Github website, they found.

See also:

• malwaretips.com

• malwaretips

• pressfrom

• fossbytes

Questions on Meta Stack Exchange should relate to features or policies that commonly apply to the network or the software that drives it.

Answer 1

Security is something that requires an architectural, holistic view.

If someone asks how to frobnicate a row in a database, we will show them how to frobnicate a row in a database and nothing more than that. If they have not secured the database (i.e.: guaranteed that only some users can frobnicate it, or made sure that scripts can't be foo'ed into it), that is another problem.

And the thing is... if we are to show them how to secure code, it's going to make answers much longer and less to the point. In some cases we might have to write the whole code the OP needs. That is not the purpose of Stack Overflow.

A good analogy would be this: if someone asks on diy.se how to splice an electric cable, and we show them, and they end up dying because they were working on a live wire, it's their fault for being a Darwin Awards contestant, not ours for not giving them a full course on how electricity works.

Answer 2

Is this an attack on Stack Overflow?

No, it isn't. You could substitute any crowd-sourced programming site for Stack Overflow in those articles, and only the numbers would change: code snippets from other sites will be found less often on GitHub, but I suspect the relative amount of vulnerable snippets is roughly the same.

If anything, this is an attack on the attitude of modern software developers. Knowing how to copy/paste does not make one a software developer; you need to know why and how the code snippet is helping to solve your problem. I can't even blame these poor souls: there's such a high demand for software developers right now that even people less suited for such tasks apply for those positions, even though they have other talents which would be much more appreciated in other kind of jobs.

Answer 3

Is this an attack on Stack Overflow?

No, this is non-news.

"The people who are using Stack Overflow, they shouldn't trust it fully," said Prof Ashkan Sami [...]

This advice should be extended to the Internet as a whole.

"It's better for programmers to do it the hard way and learn secure coding," he told The Register tech news site.

Sadly, when we get questions in php where the OP is using outdated/insecure code and we try to correct them, we get told that we're derailing the question and we shouldn't focus on that.

Damned if you do, damned if you don't.

Prof Sami said the team had developed an extension for the Chrome browser that checks when code is copied from Stack Overflow and lets coders know if it is poorly written or insecure.

This on the other hand is quite neat.

Answer 4

It's not an attack or condemnation of Stack Overflow. When new tools that give individuals amazing powers enter a market, folks tend to get a little tipsy on them. Just look what we did with jQuery <(I kid, mostly)>. We've established that Stack Overflow (and similar sites) are an essential tool for programmers in this day and age.

If you think about it, that took some work. There was a stigma surrounding asking for interactive help for programming problems, especially in academic settings. Universities put an enormous amount of effort into stopping students from using the site (some still do), but many are coming around to understanding that interactive Q&A is just a natural tool that programmers use and moving forward with that in mind.

So, that's one huge hurdle. The next one is teaching people how to use it safely, effectively, and without repercussion that ends up costing more hours to fix than time saved getting some help.

That's .. something I think we'll all be looking forward to. There's only a few more months left before the next decade rolls in. People are going to build amazing careers out of working on just that alone.

Answer 5

This is hardly news.

Just like people now have super-easy access to their friends' opinions (and vice versa) and Facebook (along with other social media platforms), instead of having to actually read the news or have a proper conversation with people… you can post your assignment on Stack Overflow and get a poorly-constructed, non-production-ready snippet in seconds.

The same thought process (read: not much of one) that led to your decision to post the question in the first place, also leads to you copy/pasting the code into your editor and away you go. There's no reasoning, no experimentation, no analysis. No drive to be better. It's too easy not to be.

Unfortunately, just as social media is having a real-world toll (see: the mental health of children, the spread of fake news, general polarization of society), the removal of this "thinking about it" barrier to entry for programming doesn't only apply to kids at school, but also to people who are allegedly actually producing software that we use every day. People who "can't do their job without Stack Overflow" (!!!) because it's "an essential tool in this day and age" (???). Of course that software is going to be riddled with holes as a result.

It's the world we live in now. Convenience has a cost, and this is it.

We did try to put a pin in it, by limiting questions to useful, thought-provoking content rather than having the site just be a code generator for the lazy. But that ship sailed long, long ago.

Answer 6

The paper's statement of

The site is popular with developers seeking advice on the best way to fix broken code.

is misleading. Although this is true, there are also many developers asking questions on Stack Overflow for help to solve general programming issues when writing new code. Using this statement, along with results from checking all of the Stack Overflow questions & answers, even those not related to fixing existing code (although in many specific cases this would not be clear), can give a distorted picture of the situation.

Stack Overflow is a Q&A site which provides answers to specific problems. Just as it's not appropriate to ask questions which are too broad or not well defined, answers should generally be focused on solving the specific question asked. It's up to the user to understand what is being provided is usually not production-ready code but, instead, just provides something which explains the solution in computer code rather than in something like pseudo code or English statements. Adding a lot of error checking in the answer can, in many cases, make the answers quite long, so it'll not only unnecessarily slow down the answerers, but also any readers who are reading them later.

Note that Stack Overflow is a teaching tool, just like programming text books and general books. The sample code in those books, from my experience, also often don't usually include a lot (if any) error type checking or handling, with them even mentioning this explicitly in some cases. The sample codes' purpose is to teach a specific concept, so their sample code focuses on just this. However, at least with physical books, users can't easily copy & paste the sample code to use it either exactly as it is, or with only a few modifications, as people can do with any code from Stack Overflow, as well as from other Web sites. I believe it's the responsibility of the programmers doing this to be aware they need to ensure the code they write not only does what they want, but also does it securely. Schools and online courses teaching programming should include the basics on how to write quality, secure code, and perhaps should also explain how to properly use sample code from various sources, e.g., books, Q&A sites, code repositories, other Web sites, etc.

Answer 7

The article is written by someone who lacks knowledge of how to actually produce software. Note that the article has no cited author, so this is largely an assumption, but for the most part the language used makes it clear there is a vast distance of misunderstanding of the way programs work when they connect security risks to not including polyfills.

If I create a piece of code to list the characters remaining in a tweet, and it is off by 1 due to reasons, that is not a security vulnerability. While there may be security vulnerabilities in a very small percent of code, the famed sql injection post comes to mind, those posts (including the sql injection one) are thoroughly debunked on the same page both in comments and answers.

Overall the majority of what you will find is straightforward answers without things like backwards compatibility or "polyfills". That is because they simply don't belong, a responsible developer is expected to include those in their public deployment as is required by their target audience and technology.

The article uses such a lack of polyfill to seemingly indicate that solutions at Stack Overflow are causing vulnerable programs (or apps as the buzz word goes).

The dangerous code chunks often used obsolete functions, did little to check user responses and did not look for attempts to break the application, said the study.

So, someone requests placing an alert from text on the screen. An answer takes some text and displays it on the screen. The answer does not prevent XSS or ill formed structure that causes the display to render funny. Is this a "dangerous code chunk"? No. It is just the most direct way to solve the problem. If there needs to be XSS support, and support for encoding etc. that would be a separate issue; one that you would both expect a developer to understand and incorporate.

I would have expected more from the BBC, although not much more. Since they are a reputable source, they should retract their low quality article. While the cited study was apparently conducted (where exactly is this study published?) by a published Associate Professor, the findings are pulled way out of proportion here.

The study indicates that there can be problems in Stack Overflow code whereby it is out dated or lacks certain error checking, which is true. The article then takes that point and makes the claim that everyone is suddenly at risk of security vulnerabilities. I wonder if there was a hidden agenda here somewhere?

"The people who are using Stack Overflow, they shouldn't trust it fully," said Prof Ashkan Sami, a computer scientist at Shiraz University in Iran who co-wrote the study.

And then, in the very next sentence, the article's author points out that the same critical "Professor" created an extension which luckily saves you from this pitfall. How convenient.

Prof Sami said the team had developed an extension for the Chrome browser that checks when code is copied from Stack Overflow and lets coders know if it is poorly written or insecure.

While it is always true that as a developer you need to ensure the code you use from third party sources is legitimate, the truth that developers should strive to avoid being lazy absolutely should not be used as a bludgeon against Stack Overflow or GitHub.

Going forward, perhaps just don't visit the BBC. I certainly don't due to a history of blunders such as this article.

There is always Reuters, the Associated Press, The Guardian, The Independent, etc. Or whatever your other favorite news sources are for that area of the world.

Answer 8

Sounds bad, right? Yes, but the paper did not look at a number of factors that potentially make this a lot less bad than it sounds.

In my experience there’s usually something that tells you when you should be skeptical of an answer: it’s score, the other answers on the question, and sometimes even the creation date are all potential signals. And the comments are even better than that as many times as they often explicitly mention what’s wrong with the answer.

(If I understood correctly, the paper found some vulnerable answers specifically because comments pointed it out.)

It’s up to the developer to listen for the signals and dig a little deeper, especially if it doesn’t point to a strong yes about the answer being good.

If anything this is an attack on laziness.

---

It’s also worth thinking about this:

• Was all the GitHub code actually vulnerable? (Were the 40% correct about their code?)
• If the code was vulnerable, does it actually matter for the project? (For example, a MOTD doesn’t need perfectly uniform randomness. And code only intended to be used by the author might not need much validation.)

---

By the way, “13% of the developers contacted”, etc. as mentioned in the news article is incorrect. It’s actually only that percentage of those who responded (=15 people), so 13% is 2 people, 40% is 6 people. Statistically this doesn’t tell us much more than very few people actually responded to them.

Answer 9

Is this an attack on Stack Overflow?

This is just sloppy work facilitated in part by the existence of Stack Overflow. Our network has on average good quality information in the highly upvoted, often visited parts but this study shows that even there not everything is really production ready.

We could warn people more explicitly about that and we could try to improve the existing information, however, the reputation system presently favors original answerers and much less editorial work, while alternative, newer answers even though they may have better momentum are often shown below older, once popular answers.

Therefore this is also an issue with the reputation and scoring system that doesn't work well for maintaining and improving aging content.

Answer 10

I'd say that to a large extent it's as much a fault of the state of education as much as anything else.

I've never been particularly good at school. When I finally got my degree - I'd had to retake one module, strategic management. Our exam was a case study and I predicted that a major GPS manufacturer would need to give up on standalone devices, and that their maps business had value, and they'd need to find new market segments. Ended up... I was right in reality.Yet I flunked the exam. It was an open book exam and I ... basically just copied my exam paper off the internet (was an open book exam) and got a Credit. I... am not proud of this.

Controversially, I believe not everyone is cut out to be a software engineer. Anyone could be and they need every opportunity to follow their desire to be one. One of the key skills I feel folks need in any field - is love for the subject, intellectual curiosity, and a deep understanding of the fundamentals. The rush to put butts in programmer and even tech seats kinda means folks are thrown into the deep end unprepared.

And yet many folks probably are encouraged to go through life this way - "write this!" but not why. Folks encouraged to just bandaid fix stuff until it works, and yet not document it. Copying code off Stack Overflow isn't just because the code is there - its because there's a culture of crunches and beating stuff into working.

If I hit my thumb with a hammer, I do not blame the hammer. I blame the person holding the hammer. That said, I also shouldn't be working on high voltage or around dangerous machinery without proper training.

Its not 'just' a software engineering problem - I posted a series of answers like this one with an alternate account because folks kept having the strange idea that kali linux makes them a hacker, without basic knowledge of the environment they work in.

So.. maybe SO might have bad code, but I feel the way things are run in the software development community doesn't equip folks to turn that 'bad' code - designed to help people understand an issue or a pattern into good code.

We have a lake. Some people fish. Some people stock the lake. A few people don't know what's the good fish, take whatver they can, and choke on it. We need to teach people not to choke, not tell them to stay away from the lake.

Answer 11

I don't think it's reasonable to consider any single website the bad guy just because people copy code from it. People will always copy code from somewhere, because there are always problems that someone can't really fix on their own, and needs outside help. They can find examples how to fix it on Stack Overflow, online forums, facebook, reddit. Even in programming books, which aren't all high quality. And sometimes the need is urgent enough that a person can't really analyze the code carefully before using it.