This paper reports a study on the effect of material surface condition on the indentation behavior of different materials, such as copper (JIS H3100 C1100), aluminum alloy (AA 6061) and stainless steel (AISI 304). The material surfaces were prepared with 1, 3 and 6 μm diamond paste, as well as P180 and P400 SiC sandpapers. The specimens made from each material had different surface roughnesses as defined with a set of parameters such as R, σ and η which were also determined. This study only managed to find the value of σ for a stainless steel specimen polished with 6 μm diamond paste. The method can, however, be used for the other two materials to obtain their σ values. It was found that a rougher surface gave rise to a higher microhardness value at low loads, which can be explained using indentation size effect (ISE) and energy approach theory. However, copper and aluminum alloy did not show such a clear trend of these effects. Stainless steel appeared to fully satisfy the ISE theory, that its microhardness was extremely high at low loads and then gradually decreased at subsequent higher loads until a constant value was obtained, which was an intrinsic hardness value of the bulk material and independent of the applied loads. The experimental results illustrated that the hardness of stainless steel varied with surface roughness. For a specimen polished with 6 μm diamond paste VHN was about 180, while for a specimen ground with P180 SiC sandpaper VHN was about 220. Copper and aluminum alloy showed consistent hardness values of VHN ≈ 85 and VHN ≈ 105, respectively, which appeared not to obey the ISE and energy balance theory at low loads.