This isn't far from how I pack for a weekend trip. I start by laying out three outfits for Friday, Saturday, and Sunday. Usually those have a top and a bottom, underwear, and socks. Friday and Sunday are the "travel" outfits, so they must be comfortable and preferably have pockets. I then add a cardigan that goes with everything in case it gets colder. Next come a spare top and bottom, also matching for everything, that account for unexpected temperature swings. So if it's summer and I've packed mostly shorts, they'll be long sleeves and pants, etc. Most of the time I know what we will be doing. In case of uncertainty, I can throw in something a little dressier. I always include an extra pair of underwear and socks. My "spare" items are usually solids in either black or blue, although I have a gray argyle cardigan that I've taken on several trips.
Then there's my travel blanket. I'll be wearing that a lot in the hotel room, so I'd rather my clothes not clash outrageously with the blue-and-white blanket. I don't have much that clashes with those colors, but I'm less likely to wear those things while traveling now than I would have been before I found the blanket.