In this paper, a fast algorithm for the simulation of deformable objects composed of multiple components made of different materials is introduced. By using the boundary element method and considering individual components and their interfaces separately, a relationship between the unknown and known displacements is established. Based on this expression, a component-based condensation method can be applied. This reduces the size of the matrix to be inverted to depend only on the number of unknown displacements of the components with changing boundary condition. To speed up the construction of the required matrices, a maximal matrix method is proposed. By categorizing the changes in boundary conditions, three fast update strategies on matrix inverse are introduced. Based on the maximal matrix method and the matrix inverse update strategy, we define eight easily formed characteristic matrices, which enhance the computation speed further. The efficiency of the proposed method is demonstrated by a set of experimental results.