According to Kevin O'Leary, people spend too much money on things they don't need – like a $2.50 cup of coffee "is such a waste of money," he says.

The smart thing to do with your money is invest it, according to O'Leary.

In fact, "the biggest money mistake people make today is they don't put aside at least $100 a week towards their retirement," he tells CNBC Make It. "That's the minimum."

While that might seem like a lot to some, O'Leary believes with the right plan, "everybody can save $100 a week."

"If you've got a job, and you're getting paid $30,000, $40,000 [or] $50,000 – if you are making that, or even less, don't spend on something you don't need" and invest instead, he says.

"The truth is, there is a lot of crap you don't need," O'Leary previously told CNBC Make It.

"What I've learned to do, and what has really helped me in maintaining growth in my own personal investing is, anytime I pick up something I'm going to buy, I say to myself, 'Do I really need this?' Because if I don't buy it, the money is going to be invested and make money every year for me while I'm sleeping."

And "the way to invest is to take $100 a week and put it to work," O'Leary says. 

To get started, O'Leary recommends using an investing app.

"There's all kinds of apps out there that help you do it right on your phone. The one that I got involved in, and I've heavily invested in, is called Beanstox," he said. Beanstox allows users to invest small amounts of money – even a few dollars – to buy fractional shares of stocks. (There is a subscription cost of $5 per month.) There are also other similar apps, like the popular Robinhood. And there are robo-advisors for beginners that provide an automated investment portfolio, like Betterment, Fidelity Go and Vanguard, also for a fee.

O'Leary recommends investing the $100 into a "diversified ETF, and let the market do its thing." (O'Leary is chairman of his own ETF company, O'Shares ETF Investment.)

An ETF, or exchange-traded fund, is a type of security that tracks an index, like the S&P 500, for example. They are "considered a lower-risk investment because of the increased diversification by holding multiple stocks or other securities," CNBC recently reported. "Some also offer full transparency by publishing their holdings each day... [and] are typically low cost." (Like any stock market investment, there are risks attached to ETF investing.)

If you invest $100 a week, "by the time you retire, in your mid 60s, you should have about $1.2 million sitting there," O'Leary estimates.

Of course, the amount you have in your 60s depends on when you start investing. If you earn a 9% rate of return, which is the average annualized total return for the S&P 500 index over the last 90 years or so, it would take 35 years to reach about $1.17 million. However, if you earned a more modest 6% rate of return, it would take about 46 years of investing $100 a week to reach about $1.21 million. Keep in mind that these numbers don't account for variables that can affect wealth over several decades, from windfalls and emergencies to rises or dips in the market, and most people diversify their retirement portfolio.

"I'm big on this financial literacy. We teach everybody everything in high school, [but] we don't teach them how to invest," O'Leary says.

"That's the big mistake [people] make – they don't invest for their future." 

Check out: The best credit cards of 2021 could earn you over $1,000 in 5 years

Don't miss:

• How Kevin O'Leary helped this small business owner recoup $200,000
• Kevin O'Leary: Why there's 'so much opportunity' to start a business right now
• Kevin O'Leary's advice for a kid running a lawn-mowing business and making thousands of dollars

Disclosure: CNBC owns the exclusive off-network cable rights to "Shark Tank." This story has been updated.