A problem

Lesson 9 of 20 in Coddy's Mathematical Riddles course.

Triangular, pentagonal, and hexagonal, (project Euler, Problem #45)

Triangle, pentagonal, and hexagonal numbers are generated by the following formula:

Triangle T(n)=n(n+1)/2 1, 3, 6, 10, 15, ...

Pentagonal P(n)=n(3n−1)/2 1, 5, 12, 22, 35, ...

Hexagonal H(n)=n(2n−1) 1, 6, 15, 28, 45, ...

It can be verified that T(1)=P(1)=H(1)=1 and that T(285) = P(165) = H(143) = 40755.

Find the next triangle number that is also pentagonal and hexagonal

Challenge

Hard

Write a function calc that solve the problem, and returns a vector of the first three solutions to the problem. It means [1, 40755, ...]

Try it yourself

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdbool.h>
#include "solution.h"

int main() {
    int rs = 0;
    int* r = calc(&rs);
    for (int i = 0; i < rs; i++) {
        if (i > 0) printf(" ");
        printf("%d", r[i]);
    }
    printf("\n");
    return 0;
}