In a medium bowl, whisk together the flour, baking soda, salt, and cocoa powder.

In the bowl of a stand mixer fitted with a paddle attachment, cream together the butter and sugar until light and fluffy, about 2 minutes. Add the egg and vanilla, and mix to combine. Then, pour in the milk and mix until combined.

Add the flour mixture, and mix just until the mixture forms a dough. Scrape the sides and the bottom of the bowl to make sure all of the ingredients are incorporated.

Roll the dough into 1-inch balls and place 2 inches apart on the prepared baking sheet. Bake for 8 minutes. While the cookies are baking, cut the marshmallows in half using a clean pair of kitchen scissors.

Remove the cookies from the oven. Place a marshmallow half, cut side down, on top of each cookie and press gently so it sticks. Return the cookies to the oven and bake for 3 minutes.

Remove the cookies from the oven and let cool for 5 minutes. Place the cookies on a cooling rack to cool completely.

When the cookies are cool, make the frosting by combining the cocoa powder, powdered sugar, melted butter, and milk in a large bowl. Mix with an electric mixer until smooth and creamy. The frosting will be runnier than frosting usually is—that is a good thing!

Place a dollop of frosting on top of each cooled cookie, coaxing it to slide down the marshmallow with a spoon or offset spatula, letting the marshmallow show through a little.

Let the cookies sit for at least 30 minutes to allow the frosting to harden. Serve with a glass of milk!