The Holes Around Mars by Jerome Bixby, Science Fiction, Adventure (Hardcover)

Allenby, in command of the "Mars I," was first to set foot outside the ship. He took his "one small step for man" -- and found himself staring at the rock in front of him. It was about five feet high. Ordinary granite -- no special shape -- and several inches below its summit, running straight through it in a northeasterly direction, was a neatly round four-inch hole.

He made a pun and grunted.

"Well, I'll be," said Janus, our photographer. "A hole."

"In a rock," added Gonzales, our botanist.

"Round," said Randolph, our biologist.

"An "artifact,"" finished Allenby softly.

Before we were done we'd found similar holes all over the surface of the planet.