I work with big data and machine learning and when I have a question for the stackoverflow community, I am often unsure of how to tractably provide a reproducible example.

With a big dataset, especially one with many columns, it may be hard to know which part of the dataset is causing the problem -- and therefore hard to know what part to reproduce.

For example, if I am trying to run a command (e.g., lasso) on a large data set (my current project's dataset is around 200 MB) but getting an error message, how could I create a toy dataset for other members of the community to reproduce that captures the necessary features of data?

I suppose one could upload a .csv to Github as discussed here. But this does not seem like the best solution to me due to privacy or proprietary issues related with many datasets.

  • 1
    I'm not sure how you could recreate such a thing. But usually it won't be necessary as long as you describe exactly what you are doing, what you've tried and the key to get a good answer : What is the error you are getting, when and where you get it. From there, chances are someone can solve your problem. Your in good hands here. – ʞunɥdɐpɐɥd May 31 '13 at 19:53
  • Contrary to what some people seem to think, an SSCCE is not always necessary. A good problem statement goes a long way. This, I think, is one of the less emphasized but more important points in question-asking tutorials; the absolute most important part is the problem statement. If you have that and some work, it's a good fit. – user206222 May 31 '13 at 21:35

You must log in to answer this question.

Browse other questions tagged .