Right click on taskbar -> properties -> on the "taskbar buttons" select the "combine when taskbar is full" or "never combine".
Thats it. Works well for me.
When you use Excel 2007 and 2010, have you noticed that all of your opened Excel files are only displayed as one window in the taskbar? This will be annoying when you tried to switch among the workbooks, and if you want to close only one file, all of the opened files will be closed at once. So how could you show all Excel windows in the taskbar?
Office Tab: Bring powerful tabs to Office (include Excel), just like Chrome, Safari, Firefox and Internet Explorer. Save you half the time, and reduce thousands of mouse clicks for you. 30-day Unlimited Free Trial
Kutools for Excel: Save 71% of your time and solve 82% Excel problems for you. 300+ advanced tools designed for 1500+ work scenario, make Excel much easy and increase productivity immediately.60-day Unlimited Free Trial
In Excel 2007 and 2010, you need to check an option in the Excel Options to solve this task.
1. Click File > Options in Excel 2010 or click Office button > Excel Options in Excel 2007, in the Excel Options dialog box, click Advanced from the left pane, and then scroll down to Display section, and check Show all windows in the taskbar option, see screenshot:
2. Then click OK to close this dialog, and all the opened workbooks have been displayed in the taskbar, you can switch among them quickly, see screenshots:
Note: In Excel 2013, this Show all windows in the taskbar feature has been removed from the Excel Options.
If you have installed Office Tab in your computer, it helps you view and edit multiple files within a single tabbed window, to know more about Office Tab and want to down load it please click here.
After installing Office Tab, open multiple workbooks, you will see all the workbooks name have been displayed within a single tabbed window as following screenshot shown:
And if you also want to show these windows in the taskbar, please click Office Tab > Options Center, in the Tabs for Excel Options dialog, check Display all windows in the taskbar under General & Position tab, see screenshot: