Place cubed potatoes in cold, salted water and bring to the boil. Reduce heat to a simmer and cook until just cooked, about 10 minutes. Then drain and allow to dry, before mashing.
In the meantime (if not using left-over baked/ grilled fish), poach the fish. Place fish in a pan and cover with salted water. Bring to a gentle boil, and allow to simmer for about 5 minutes (depending on the thickness of the fillet) until just cooked. Drain the fish from the water with a slotted spoon, remove skin and, using a fork, flake into large chunks. Discard any bones.
Place the fish, potatoes, chili, scallions, and cilantro in a bowl, add a squeeze of lime and season with salt and pepper. Use your hands to combine the mixture. Then divide and shape mixture into 4 to 6 even-sized fish patties and place on baking sheet lined with foil
To bake: Pre-heat oven to 400°F (200°C). Bake for 25 to 30 minutes until golden-brown and crisp. Midway through, turn over and brush tops with a little more olive oil.
To broil: Pre-heat broiler. Place baking sheet about 4 inches below the heating element. Broil for 5 to 6 mins each side until nicely brown on both sides and hot through. After flipping, brush top with more olive oil.
Serve with lime wedges.