What Did Sailors Use Prior To The Invention Of The Sextant?

The navagational technique used before instruments was called "Dead reckoning"  

From en.wikipedia.org  

Dead reckoning (DR) is the process of estimating one's current position based upon a previously determined position, or fix,
and advancing that position based upon known speed, elapsed time, and
course. While traditional methods of dead reckoning are no longer
considered primary in most applications, modern inertial navigation systems, which also depend upon dead reckoning, are very widely used.