According to the United States Census Bureau, the CDP has a total area of 2.0 square miles (5.2 km²), all of it land.
As of the census of 2000, there were 4,474 people, 1,137 households, and 966 families residing in the CDP. The population density was 2,228.5 people per square mile (859.4/km²). There were 1,211 housing units at an average density of 603.2/sq mi (232.6/km²). The racial makeup of the CDP was 47.41% White, 0.45% African American, 1.56% Native American, 0.63% Asian, 0.04% Pacific Islander, 40.43% from other races, and 9.48% from two or more races. Hispanic or Latino of any race were 76.15% of the population.
There were 1,137 households out of which 55.3% had children under the age of 18 living with them, 61.8% were married couples living together, 16.2% had a female householder with no husband present, and 15.0% were non-families. 11.6% of all households were made up of individuals and 4.7% had someone living alone who was 65 years of age or older. The average household size was 3.93 and the average family size was 4.22.
In the CDP the population was spread out with 37.8% under the age of 18, 12.4% from 18 to 24, 28.7% from 25 to 44, 15.3% from 45 to 64, and 5.7% who were 65 years of age or older. The median age was 25 years. For every 100 females there were 112.3 males. For every 100 females age 18 and over, there were 107.2 males.
The median income for a household in the CDP was $26,052, and the median income for a family was $26,166. Males had a median income of $20,410 versus $19,583 for females. The per capita income for the CDP was $9,101. About 25.6% of families and 30.8% of the population were below the poverty line, including 41.8% of those under age 18 and 10.3% of those age 65 or over.
In the state legislature Ivanhoe is located in the 16th Senate District, represented by Democrat Dean Florez, and in the 34th Assembly District, represented by Republican Bill Maze. Federally, Ivanhoe is located in California's 21st congressional district, which has a Cook PVI of R +13 and is represented by Republican Devin Nunes.