I just got back from climbing Mt. Langley, and have hiked the area a lot over the years. I would agree that taking a three year old to the Cottonwood lakes would be a lot to ask on a day hike. My eight year old sons' first real backpack trip, where he was hiking, was when he was almost 4. We went over Trail Pass to Mulkey meadows. It was about a 5 mile hike in, that took a LONG time with all the breaks. If your three year old is fine with riding in a carrier, and you can handle the weight and distance, then I'd say "go for it". If you expect him to walk most of it, I'd say "go check out the old cabin off the trail towards Cottonwood Pass".