The advent of Multi-core processors has offered powerful processing capabilities and provided new avenues for parallel processing. As the traditional methods of computation are inherently based on single core of execution, they are not capable of taking the advantage of high computational power offered by multi core processors, even if available. Singular Value Decomposition (SVD) has been proven well for many image processing techniques such as image compression, quality measure and in watermarking. SVD is a highly compute intensive algorithm that applies numerous matrix operations to an image such as transpose, inverse, multiplication of high orders to form a compressed image. However accelerating the SVD routines to exploit the underlying hardware poses a significant challenge to the programmers. This paper deals with improving the speedup of SVD algorithm used in image compression technique. This is achieved by identifying the areas where data parallelism can be applied and parallel programming model OpenMp is used to run the parallel components on multiple processors. This results in faster execution of the algorithm and hence reduces the execution time when applied to image compression for large images. This paper also addresses the space overhead of storing the matrices of SVD technique, by adapting efficient sparse matrix storage format such as Compressed Row Storage (CSR). Experimental results show that the speedup achieved through OpenMp is around 1.15 and better compression ratio with sparse matrix storage format.