Using the Medical and Surgical Histories, this is a chart of the Aggregate Present in the Atlantic Region (including the Dept of Western Virginia, transferred in August 1864). "Grant" on the graph is those forces actively engaged with the enemy (the Depts of the Potomac, Virginia and Western Virginia), whilst "Garrison" is all the other Depts. Colored troops were only returned by Region and so can't be assigned.
As you can see, Grant lost a gross of over 90,000 men in April to August 1864. Some of these are battle casualties, some are sick and some are discharged. However, this is already offset by new recruiting somewhat, so the actual loss to the army is considerably greater.