As a man who is taking his first boat out this spring, don't buy a really nice one until you learn boat control.

pick up a cheap 12-14 with a 9 hp and practice fishing at a lake get used to it before making the plunge.  you will always get the money back out of a small boat  if you like it then up grade.  boating is like ice fishing you have to be careful.

Depends how old your kids are I guess. Generally the deeper the better if they are still quite young. Also the easier the boat is to control while fishing, the better experience you will have. Also make sure you get a good trailer, the easier to load the better. There's nothing worse then having to work to fish, by that I mean a bunch of monkey business, the boat won't start, or you can't control it in the wind it's a challenge to load, it just ruins the experience and as a first time owner you will want to make things as easy as possible. For a first timer I would recommend something with a roller trailer, they basically load themselves right up. A good trolling motor on the front goes a long ways nowadays, you can't go wrong with them. Make sure the boat isn't under powered, it just may leave a bad taste in your mouth. get a good depth finder, make sure the boat has a live well. Also, make sure you get some good marine batteries. Life jackets for everybody etc....Good tires on the trailer, lights working.

I would try to get the most bang for your buck.  Depends on what water you will be fishing and how you plan to fish.  I like the safety and little bit smoother ride of glass, but am not apposed to metal.  Also, think about getting one with a big enough motor to pull a tube so you don't burn out the kids.  I think it's good to mix it up a bit when fishing stinks and the weather is hot.  Sometimes it's hard to do, but you got to keep them kids having fun. 

I would mainly be fishing local lakes. Not the river or anyplace such as Sakakawea or Devils lake.  The kids are 9 and 11, but also have a 2 year old. The little one probably would not be going with unless mom came along. So if that were the case would like to be able to take all 5 of us with so we can all enjoy the time out and together.

start small, as others have suggested.  After a year or two, you'll get a better idea of what you like/dislike and what your actual needs are before/if you decide to upgrade.  Some items to consider:  1)  where are you going to store it in the offseason?  2) how frequently do you plan to use it?  3) think about the "accessories" - the cost adds up... 

The reason you see so many used boats for sale in the spring is the same reason you see so many permanent fish houses for sale in the fall.  They are sometimes more work than what one expects/wants to put into it....

What the heck is with the start small comments for a person needing to bring 2-4 people along with him?

I say buy a boat big enough for the family.  A 17-18 ft boat that is 80+ inches wide.  Lots of good ones out there in that category that can be had in a wide variety of price ranges.

A 12-14 ft boat with a 9.9 motor can put fish in the well on most any lake or river, but unless you are a dunce (and I don't think you are), you will learn just as quickly how to handle a bigger boat with a tiller motor, side console, or a full windshield.  Buy the boat that fits your needs now keeping within your budget.  Doesn't have to be that new $60k Lund or Ranger, but an older Lund, Ranger, Crestliner, Alumacraft, etc, etc are all good rigs.  The key here is to get one "new" enough to be dependable since you will want to make sure it gets you back to the dock. 

What kind of budget are you working with?  Not to be nosey but it will help the many on here determine what your options are within what's available.

One of the key things here that others seem to have ignored is that you have friends with boats.  They should be able to help guide you as well in your boat buying decisions, AND help you get up and running on the water by acting as a co-pilot for a trip or two.

Other than that, I just say once you get a boat to go easy at first.  We who have had boats for years can put them in and take them out of the water in a couple minutes, if you are at a busy ramp don't be afraid to ask for help!  Many great people out there will help you load/unload but there are always a few who seem happier to sit and whine about your as yet undeveloped dock skills.

How much money are you thinkin of spending?

p.s.  Good idea already to keep the 2 yr old off the boat unless mom comes with.  Watching over a 9 and 11 yr old in the boat is enough to keep you busy without having to worry about a toddler.

Sounds like a 16 or 17ft aluma-craft tiller would fit the bill.  I have a few friends who have had this or are currently running them and they have had good luck with them.  I was even out in one on Devils in 4 footers one time.  I wouldn't reccomend it, but thank god for Gore-Tex. 

Tillers will give you the best bang for your buck and are nice and open and that size would probably be easy to load on smaller lakes.  With kids that age, I don't think them going over board would be a big issues, but being on windy big water could scare them and you don't want that.

I'd look into the classifieds/dealers in Minnesota.  They have a number of small lakes and it seems boats like this are everywhere, so you might be able to find a good used one.

Another boat to look at is a 17 or 18ft Tyee.  They are probably the bread and butter of family fishing.  They may cost a little more, but would be worth looking into.

Send me a pm if you want to talk boats and I can try to help you find something that fits your needs. I work with a dealership, not for one so I've got a pretty good knowledge of boats.

I would go with something like this.  Small enough to tow and handle fairly easy, yet large enough to accomodate your family.  Plus, it won't break the bank.

https://www.walleyecentral.com/classified/new/adviewer.jsp?adv=16368

Going to have to agree with Allen

DONT start small!!
DO start used!!

Buy a foot bigger than you think you need, you always will need the space. The difference between driving a big boat and a little boat aren't that much difference, however backing down with a little boat can cause you to go nuts, bigger boats are easier to back IMHO. Pontoons are the exception to the rule ,they are just weird!!!! LOL

Buy used not new ,whatever you end up buying, the odds of you keeping it if the family really gets into is slim to none. You are going to find things that you now need and others you dont, some things you wont be able to live without. You can unload and shouldn't get to tore up on price when you sell and you will have a good idea with what you want in your next boat. Once you buy your next boat always, always, max out the horese power if you are talking under 200 HP.

Captain Ahab Said:
I would go with something like this.  Small enough to tow and handle fairly easy, yet large enough to accomodate your family.  Plus, it won't break the bank.

https://www.walleyecentral.com/classified/new/adviewer.jsp?adv=16368

Only problem with this boat is it is a southern boat, which means it doesnt get the 4-6 month break from the lake like the boats around here do, it gets twice the use, most likely anyway.