Renewable energy sources (RES) are essential in the generation of electrical power, especially for a grid-connected system. The power quality issues arise due to the integration of the RES with a grid. Due to the power quality issues, the grid experiences problems such as voltage sag, swell, harmonics, and load shedding that affect the consumers. To overcome the abovementioned problems, custom power devices (CPDs) are employed in the distribution system. A new FACTS-based distributed power flow controller (DPFC) that incorporates a coordinated PQ theory and a FOPID controller is proposed to mitigate the power quality issues. The DPFC is composed of a shunt, and a series controller mitigates sag, swell, and current harmonics that exist in the distribution system. In this study, solar and wind energy systems are considered as the source. For the validation, the results of the proposed FOPID are compared with PI, FUZZY, and ANFIS controllers. The FOPID controller shows a better voltage profile improvement in terms of the amplitude peak of transition rise/fall voltage and in compensation of the voltage with fewer oscillations. In addition, there are also fewer ripples in the DC voltage, and the improvement of active power is implemented in the proposed DPFC-FOPID controller. Furthermore, a case study was conducted on an IEEE 12 bus system using CPDs. The study showed that the developed DPFC exhibits superior performance in terms of voltage compensation and harmonics reduction. This paper presents a DPFC-based integrated hybrid system model using a fractional order PID (FOPID) controller under the unbalanced voltage conditions in the MATLAB/Simulink environment. © 2020 Elsevier Ltd