Cfrtjryk

I'm a sociology undegrad working on an essay for a methods class. I'm also planning on submitting it as a sample for my application to grad school. I don't want to be too specific, but I believe that this work is quite original and my hypothesis would confirm previous literature, and all in all I think it would would make a good impression on the admissions committee.

So basically I've run the tests and I'm getting conflicting results. Using one dataset (which has more observations) gives me very significant results, while using another one (which would arguably be more accurate) doesn't give me anything. So here I am at a crossroads, and I've come up with three possible options as to what to do:

1. Only show the significant results. After all, this is just a ten-page essay, it's not supposed to be publishable or anything, right?




2. Only use the better dataset and admit that there just isn't much there - maybe blaming it on the small sample size or on the not-so-good dependent variable. Hopefully the committee would appreciate the honesty and the relatively advanced methods that I used.




3. Show results from both datasets, suggesting that the differences might be due to the sample size or maybe to chance.

As I type this I'm leaning more towards option 3, but I'd like to hear from people with more experience in academia. What should I do?

In research, you don't set out to prove that something is true. You set out to discover whether or not it is true. This would be knowledge. The other is just propaganda.

Negative results are not a failure. They give you evidence just as do positive results. If you ignore, or obscure, results you are lying to yourself and others. If you design an "experiment" so that it is guaranteed a priori to produce positive results, it isn't research.

Hoping that something is true isn't evidence. Many researchers start out with that idea. I think this is true. I really want it to be true. But if it is false, it is just as valuable (possibly more so) to know that and to be able to investigate why.

Report all your results. Try to explain why different aspects lead you in different directions. Only then can your learning begin.

Omitting negative findings and selectively reporting only the positive findings would be a breach of research ethics. As a researcher you are supposed to uncover knowledge,* not to obscure it. Findings are often contradictory and in need of interpretation. By explaining how you obtained these contradictory results (i.e. your methods), you help others to avoid dead ends in the future and to make sense of what looks confusing today.

_{*Interestingly, the knowledge that research creates often takes the form of higher-level confusion rather than ultimate certainty.}

Are your significant results a large effect size, or just a tiny change that is significant because of the large sample size?

Are your non-significant results similar in direction and magnitude to the significant results from the other dataset?

Consider how much the size of the dataset is impacting what you are seeing - you may be able to frame one study as confirming the results of the other if they are in agreement apart from significance. Look at more than just the p-values, especially if they are coming from a very large dataset.

How honest should I be in disclosing not-so-exciting results?

You should always be completely honest: Show the results of both datasets and let the conclusion follow from the data. Comment objectively on the quality of the two datasets, and their sample sizes, but don't exclude data merely because it gives undesirable or unexciting results. In terms of the differences between the datasets, if you know why they are different then explain this, and if you don't know why they differ, then say so - don't present your speculations as scientific conclusions.

I'm only a student too (graduate level), but here are a couple more reasons to go with option 3 of showing both data sets:

• As mentioned in henning's comment, perhaps you can use your unusual results as a stepping stone for further research, and include this in your application. Treating unsatisfactory results in such a way can show that you have motivation and resilience.




• If you did good work and showed it, even without getting "good results", that can show that you at least have potential.




• Furthermore, in the context of applications where people usually put only their best foot forward, your honesty may actually be appreciated and respected by the admission committee. It can show that you put science first.

For option (3), add 'or there is something I do not yet understand going on".

This is much more interesting.

Your undergraduate course is there to teach you how to answer questions.

The important thing in research of any discipline is not getting the right answers but asking the right questions.

So, present both data sets, call out the discrepancy and try to explain why that is interesting and why it is worth following up.

Setting out a mini research problem like this could make you stand out much more than simply having a result.