A self-parallelizing computer system and method asynchronously processes execution sequences of instructions in two modes of execution on a set of processing elements which communicate with each other. Each processing element is capable of decoding instructions, generating memory operand addresses, executing instructions and referencing and updating its own set of general purpose registers. These processing elements act in concert during the first mode of execution not only to execute the instructions in an execution sequence but also to partition an execution sequence into separate instruction subsequences. The separate instruction subsequences are stored along with additional information which will allow the stored subsequences to be correctly executed in parallel. Subsequent re-execution of the same execution sequence is done much faster in the second mode of execution, since each of the processing elements decodes and executes only the instructions in one of the subsequences while the other processing elements are concurrently each doing the same with another one of the subsequences.