There’s no right place to start in getting your business going. What I sometimes see though, is people spending too much time in the planning phase, hoping to get the design perfect. I’m a big time advocate to think about the smallest possible version of your business and use that as a ‘prototype’ to start testing your business in the real world. You will learn so much more by doing. It’s scary, I know, but hugely valuable.
As for the bookkeeping tools, I’ve recently switched to Wave myself and have been enjoying that tool. It’s free! And works across devices. You can link your bank account and it’ll important transaction for you. Super easy and straight forward.