InceptionTime Model for Time Series

[codeboy5]

This PR will contain the implementation of InceptionTime Model and it's use for classification and regression task.

Some of the commits from the PR #253 are also in this PR, but will take care of them when that PR is merged.

[codeboy5]

Have to fix a couple of issues with the residual part.

[codeboy5]

Hey @ToucheSir @darsnack @lorenzoh,
So If i do use the residual layers, I seem to be getting a size mismatch during the backprop. The forward pass and loss computation works fine. I am still trying to figure out where exactly the issue is. If you see any issues by just taking a quick look at the code, it would help. Thanks

[ToucheSir]

Try building up your model bit-by-bit and checking for size mismatches on each submodule with a dummy input. Flux.outputsize can also be very useful here.

[codeboy5]

Try building up your model bit-by-bit and checking for size mismatches on each submodule with a dummy input. Flux.outputsize can also be very useful here.

Yeah I tried doing that. I dont understand how the forward pass works fine but the backprop gives an error. Will figure it out.

[ToucheSir]

Since only InceptionModule has been ported over so far, you could save some time by just testing with that. Feel free to drop a MWE and stacktrace here if you get stuck.

InceptionBlock will need to receive the same treatment at some point. One resource which might help are the 2D Inception image models in Metalhead. Those make extensive use of Flux container layers.

[codeboy5]

The code can be tested with the following snipet :-

data, blocks = load(datarecipes()["natops"]);
task = TSClassificationSingle(blocks, data);
traindl, validdl = taskdataloaders(data, task, 16);
callbacks = [ToGPU(), Metrics(accuracy)];
model = FastTimeSeries.Models.InceptionTime(24, 6);
learner = Learner(model, tasklossfn(task); data=(traindl, validdl), optimizer=ADAM(), callbacks = [ToGPU(), Metrics(accuracy)]);
fitonecycle!(learner, 10, 0.01)

[ToucheSir]

Just a couple comments on the WIP here. Mostly looks good and I'm glad you can get it training :)

[ToucheSir]

I think it makes more sense to have Conv1d be lowercase conv1d, but that's a minor thing.

[codeboy5]

Hey guys, so the model is working for regression task too now, will push the notebook in a while.

[ToucheSir]

I have some suggestions for the notebook, but that can wait for a follow-up PR. Will try to have a look at the model again in the next couple of days.

[ToucheSir]

Still many wild ranges floating around ;)

[codeboy5]

Yeah, will fix that in the next commit.

[ToucheSir]

Mostly LGTM once docstrings are added and commented code is restored/removed.

[codeboy5]

Mostly LGTM once docstrings are added and commented code is restored/removed.

Hey, will push the docstrings today. This PR should be ready to go after it unless we want to further improve something. We can also discuss the future steps in our next biweekly.

[ToucheSir]

Sorry about the delay on reviewing this. Let's get it merged.