What is the difference between Bayesian and frequentist statistics?
What probability means. To a frequentist, probability is the long-run frequency of an event over many repetitions. To a Bayesian, it is a degree of belief, which can be updated as evidence arrives. Almost every practical difference follows from that.
The frequentist approach. Parameters — the true effect size, the true rate — are fixed but unknown. Probability statements attach to the data, not to the parameter.
So a frequentist can say: "if there were no effect, data this extreme would occur 3% of the time". They cannot say "there is a 97% probability of an effect", because the parameter is not random.
This is exactly why p-values and confidence intervals have their awkward definitions — the intuitive interpretation people reach for is Bayesian, and the framework does not support it.
The Bayesian approach. Parameters are treated as uncertain, described by probability distributions. You begin with a prior — what is believed before this data — combine it with the likelihood of the observed data, and obtain a posterior: updated belief.
The output is directly interpretable: "given this data and these assumptions, there is a 97% probability the effect is positive". A credible interval means what people wrongly assume a confidence interval means.
The long-running objection to Bayesian methods is the prior: it introduces something outside the data, and different priors give different conclusions. The response is that assumptions exist in every analysis, and a stated prior is more honest than a hidden one — plus with sufficient data the prior's influence diminishes, and different reasonable priors converge.
The practical position now. The philosophical argument has cooled considerably. Most working scientists use frequentist methods because they are standard, expected by journals, and computationally simple. Bayesian methods have become far more usable with modern computing and are standard in some fields.
Where Bayesian methods are clearly advantageous: incorporating genuine prior knowledge; sequential analysis, where data accumulates and can be assessed as it arrives without the penalties frequentist methods impose; complex hierarchical models; and situations where a direct probability statement is what decision-makers actually need.