What is machine learning regression
Difference between linear regression in machine learning and the statistical model
I had the understanding that the main difference between machine learning and the statistical model is that the later "assumes" a certain kind of data distribution and is based on this different model paradigm as well as statistical results that we get (e.g. p-values, F-statistic), t-stat, etc.). In machine learning, however, we don't care about the distribution of data and we are more interested in predictions.
While looking through mllib doc, I found that we are giving a distribution for linear regression. But Mllib is a machine learning package. So I have the following questions:
1) Is my understanding between ML and statistical method wrong?
2) Does Spark use statistical models for linear regression and GLMs?
Note: There are many wonderful posts on the difference between machine learning and statistical methods. However, this is more related to the MLLIB spark.
- Unfortunately, the dichotomy you have described is invalid. ML models (almost always) define a distribution of responses. For example, the extremely popular XGBoost machine library defines certain learning objectives (e.g. linear, logistic, Poisson, Cox, etc.) to increase the gradient.
- The implementation of linear regression and GLMs in Spark's MLlib is based definitely on standard statistical theory for linear models. For example, quote directly from the comments: This is a standard linear regression algorithm for the Gaussian answer. The implementation of each algorithm could be tweaked to work for very large amounts of data (see for example this excellent thread on "Why use gradient descent for linear regression when a closed-form math solution is available?"), But the one behind it standing theory An algorithm is exactly the same.
- Why do Japanese stare at foreigners
- What is the definition of femininity
- Do forest fires contribute to climate change
- What makes a question interesting
- What is the average website traffic
- Are there alternatives to target stores
- How do you distract yourself from loneliness
- Smart people make drugs
- What if incest was legal?
- Hanukkah is an important Jewish holiday
- Who should I buy Christmas presents for?
- Anxiety is the root of Obsessive-Compulsive Disorder
- How many cities does Mozambique have
- How are sex offenders treated in prison?
- What are the applications of differential equations
- Communism destroys human greatness and achievement
- How much does a crore cost in GBP
- Is the minimum wage constitutional
- What is a metric fox
- Is Donald Trump really a nationalist?
- Will Puigdemont rule the independent Catalonia of Brussels?
- What's the healthiest way to die
- How do I search for TV scripts
- How popular Quorans plan their responses