Indian Banking system has historically evolved under the impetus of economic, social and institutional forces. Public sector banks act as backbone of economic growth, prosperity and they support the implementation of successive five year plans by the government. During the post nationalization era, Public Sector Banks PSBs faced serious erosion in the productivity and profitability due to tough competition among the different bank groups. The overall operational efficiency of the banking system depends on the relative efficiency of each unit of the banking system. To assess the relative performance of each public sector banks in relation to the Operating efficiency, the Herfindhal’s index of concentration has been computed using few selected parameter. The five parameters used are: Net Profits, Total income, Total Expenditure, spread and Burden. This index was calculated for each State bank of India 1 and its associates 7 and nationalized banks 19 on Absolute Volume concentration index for the period of 18 years as first generation reforms 1991 to 1997 and the second generation reforms 1998 to 2008. This paper highlights the financial health of PSBs in selected parameters to find out the necessity to rejuvenate the Indian banking sector as it happened in the U.S. banking industry due to the Subprime crisis. It also helps the State bank of India and its associates as well as nationalized banks to improve the Interest income and Non-interest income and reduce the Non-performing assets, investing in free risk securities etc., in order to increase the profit margin to meet the competitions from the private and foreign banks.