解:延长$AD$、$BC$交于点$E。$
$\because$ 四边形$ABCD$内接于$\odot O,$$∠ A=90°,$
$\therefore ∠ BCD=90°,$$∠ EDC=∠ ABC。$
在$\mathrm{Rt}△ ABE$中,$\cos B=\frac{AB}{BE}=\frac{3}{5},$$AB=17,$
$\therefore BE=\frac{5}{3}AB=\frac{85}{3},$
由勾股定理得$AE=\sqrt{BE^2-AB^2}=\sqrt{(\frac{85}{3})^2-17^2}=\frac{68}{3}。$
$\because ∠ EDC=∠ ABC,$$∠ ECD=∠ A=90°,$
$\therefore △ EDC ∽ △ EBA,$
$\therefore \frac{CD}{AB}=\frac{DE}{BE},$即$\frac{10}{17}=\frac{DE}{\frac{85}{3}},$
解得$DE=\frac{50}{3},$
$\therefore AD=AE-DE=\frac{68}{3}-\frac{50}{3}=6。$