A horizontal bore hole is sidetracked through a window cut in a cased vertical well or from a vertical open hole shaft extending below the kickoff point. In one embodiment, a whipstock is used. In another embodiment, the cased vertical well provides a drillable joint so the window can be cut with a conventional bent housing mud motor from a cement plug located adjacent the drillable joint at the kickoff point. In yet another embodiment, a cement plug is dressed down to the kickoff point in a vertical open hole and is used to start the curved well bore. After drilling at least the curved bore hole, a production string extending into the vertical well is cemented in the curved bore hole and then cut off inside the vertical cased hole with a conventional burning shoe/wash pipe assembly. The whipstock or cement plug is removed to clear the vertical well to a location below the entry of the horizontal well bore. Multiple horizontal wells may be drilled. Any open hole portions of the vertical well are cased with a liner. A downhole pump may be provided in the vertical well below the entry of the horizontal well bore. In addition to one or more horizontal completions, one or more productive intervals can be perforated through the vertical well to provide vertical completions.