Prepare cupcakes according to box’s directions. Allow the cupcakes to cool completely.
Add butter, powdered sugar and 4 tablespoons milk to a large bowl. Use an electric mixer to beat together all ingredients, starting on low and gradually increasing to high. Add more milk if frosting is too thick.
Beat ingredients until smooth, creating semi-stiff peaks.
Divide buttercream into 4 separate bowls. Add one color of gel food coloring to each bowl. To achieve a rich color, add about 4-6 small drops (a bit less than 1/2 a teaspoon).
Mix each bowl of buttercream until completely tinted.
Attach Wilton icing tip #21 to a piping bag. Scoop colored buttercream into piping bag, alternating colors.
Frost each cupcake.
Decorate each cupcake with cereal (making sure to use plenty of the marshmallow shapes).