The life cycle of android works like that.You don't have to exit the application.When user clicks on the home button all activities for that application will still be there on the activity stack.Android automatically finishes them whenever it requires additional memory or after a span of time.
If you want start with the root activity every time you can use android:clearTaskOnLaunch=true.
The java system.exit call would be brute forcing it, and inelegant in my opinion. And the Android lifecycle read is important but it doesn't address when the developer would like an explicit "Quit" or "Exit Application" option.
Call finish() from your root activity. This is the activity that you've first entered in your app execution. This should finish the child activities as well.