Recently Tao Guo (1999) proposed a stochastic search algorithm in his PhD thesis for solving function optimization problems. He combined the subspace search method (a general multi-parent recombination strategy) with the population hill-climbing method. The former keeps a global search for the overall situation, and the latter maintains the convergence of the algorithm. Guo's algorithm has many advantages, such as the simplicity of its structure, the high accuracy of its results, the wide range of its applications, and the robustness of its use. In this paper a preliminary theoretical analysis of the algorithm is given and some numerical experiments are performed using Guo's algorithm to demonstrate the theoretical results. Three asynchronous parallel algorithms with different granularities for MIMD machines are designed by parallelizing Guo's algorithm.